mlsanityData to 7 Oct 2026

GutenOCR: A Grounded Vision-Language Front-End for Documents

arXiv 2601.14490 ↗First posted 20 Jan 2026cs.CV cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

GutenOCR is a family of grounded OCR front-ends obtained by fine-tuning Qwen2.5-VL-3B and Qwen2.5-VL-7B. The resulting single-checkpoint vision-language models expose reading, detection, and grounding through a unified, prompt-based interface. Trained on business documents, scientific articles, and synthetic grounding data, the models support full-page and localized reading with line- and paragraph-level bounding boxes and conditional ``where is x?'' queries. We introduce a grounded OCR evaluation protocol and show that GutenOCR-7B more than doubles the composite grounded OCR score of its Qwen2.5-VL-7B backbone on 10.5K held-out business and scientific pages (0.40 to 0.82). On Fox and OmniDocBench v1.5, our approach substantially improves region- and line-level OCR as well as text-detection recall, but reveals trade-offs in page-level linearization, color-guided OCR, and formula-heavy layouts.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

36 of 54 references.

References · 36

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  5. 2021Grounded Language-Image Pre-trainingcited by 1,147
  6. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  7. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  8. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  9. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
  10. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  11. 2021OCR-free Document Understanding Transformercited by 400
  12. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
24 more
  1. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  2. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  3. 2017EAST: An Efficient and Accurate Scene Text Detectorcited by 248
  4. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
  5. 2019Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapescited by 171
  6. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  7. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  8. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  9. 2018FOTS: Fast Oriented Text Spotting with a Unified Networkcited by 109
  10. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 108
  11. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  12. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  13. 2016Detecting Text in Natural Image with Connectionist Text Proposal Networkcited by 85
  14. 2021LayoutReader: Pre-training of Text and Layout for Reading Order Detectioncited by 79
  15. 2024Focus Anywhere for Fine-grained Multi-page Document Understandingcited by 62
  16. 2025Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversioncited by 58
  17. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 51
  18. 2025olmOCR 2: Unit Test Rewards for Document OCRcited by 41
  19. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 39
  20. 2023Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Predictioncited by 23
  21. 2024DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understandingcited by 23
  22. 2020CLEval: Character-Level Evaluation for Text Detection and Recognition Taskscited by 10
  23. 2025'Eclair -- Extracting Content and Layout with Integrated Reading Order for Documentscited by 9
  24. 2024Large Language Models for Page Stream Segmentationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Chronos: The AI Co-Historiancited by 0
  2. 2026Impact of Iterative Fine-Tuning on Transcription Accuracy in Complex Historical Sanskrit Manuscriptscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks
  2. 2026LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR
  3. 2026Qianfan-OCR: A Unified End-to-End Model for Document Intelligence
  4. 2025DRISHTIKON: Visual Grounding at Multiple Granularities in Documents
  5. 2026From Retrieval to Recognition:How Vision--Language Models Become OCR Specialists
  6. 2026Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models
  7. 2023Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs
  8. 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Models