Omnidocs Visual Document Toolkit Launched
A new unified toolkit for visual document processing named Omnidocs has been launched. The tool is designed for RAG-like tasks on complex documents and supports serving frameworks like vLLM and PyTorch. It can run on consumer-grade GPUs or Apple Silicon-based Macs.
- The toolkit is developed by Adithya S K, who is also behind other open-source projects like VARAG (Vision-Augmented Retrieval and Generation), a RAG engine that prioritizes vision-based retrieval techniques. - Omnidocs provides a consistent `.extract()` API for a variety of document analysis tasks, including layout detection, OCR, table parsing, and structured extraction, aiming to simplify the development workflow. - Its multi-backend architecture allows for flexibility in deployment, supporting PyTorch for local GPU use, MLX for Apple Silicon, and vLLM for production environments, which can streamline the transition from development to production. - A key feature is the VLM (Vision-Language Model) API that does not require a local GPU, enabling the use of models from providers like OpenRouter, Gemini, Azure, and OpenAI. - The toolkit is designed to facilitate advanced RAG techniques that go beyond traditional text-based methods by allowing for the direct processing of visual information in documents, which can prevent information loss that sometimes occurs with OCR. - For structured data extraction, Omnidocs can directly populate Pydantic schemas, ensuring type-safe and structured output from visual documents. - The roadmap for Omnidocs includes plans for more advanced features like math recognition for LaTeX extraction and a deeper understanding of charts.
