mlsanityData to 14 Oct 2026

Qianfan-OCR: A Unified End-to-End Model for Document Intelligence

arXiv 2603.13398 ↗First posted 11 Mar 2026cs.CVcited by 21

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

We present Qianfan-OCR, a 4B-parameter end-to-end vision-language model that unifies document parsing, layout analysis, and document understanding within a single architecture. It performs direct image-to-Markdown conversion and supports diverse prompt-driven tasks including table extraction, chart understanding, document QA, and key information extraction. To address the loss of explicit layout analysis in end-to-end OCR, we propose Layout-as-Thought, an optional thinking phase triggered by special think tokens that generates structured layout representations -- bounding boxes, element types, and reading order -- before producing final outputs, recovering layout grounding capabilities while improving accuracy on complex layouts. Qianfan-OCR ranks first among end-to-end models on OmniDocBench v1.5 (93.12) and OlmOCR Bench (79.8), achieves competitive results on OCRBench, CCOCR, DocVQA, and ChartQA against general VLMs of comparable scale, and attains the highest average score on public key information extraction benchmarks, surpassing Gemini-3.1-Pro, Seed-2.0, and Qwen3-VL-235B. The model is publicly accessible via the Baidu AI Cloud Qianfan platform.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 24 references that are in the map. First posted on arXiv 11 Mar 2026.

Builds on

22 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models25 Feb
  2. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models13 May

Other references · 20

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  6. 2019Towards VQA Models That Can Readcited by 1,718
  7. 2019Root Mean Square Layer Normalizationcited by 1,480
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  11. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  12. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
8 more
  1. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  2. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  3. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  4. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 150
  5. 2025PaddleOCR 3.0 Technical Reportcited by 143
  6. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 108
  7. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 57
  8. 2024BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Taskscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  2. 2026Unlimited OCR Workscited by 8
  3. 2026OvisOCR2 Technical Reportcited by 5
  4. 2026How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settingscited by 4
  5. 2026TexOCR: Advancing Document OCR Models for Compilable Page-to-LaTeX Reconstructioncited by 1
  6. 2026MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsingcited by 1
  7. 2026LayoutLite: Token-Level Implicit Layout Analysis for Efficient Document OCRcited by 1
  8. 2026When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documentscited by 1
  9. 2026PaDoc: Layout-Grounded Parallel Decoding for Document Parsingcited by 1
  10. 2026All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Expertscited by 1
  11. 2026Edges Before Embeddings: A Confidence-Aware Blur Gate for Vision-Language Pipelinescited by 0
  12. 2026HPD-Parsing: Hierarchical Parallel Document Parsingcited by 0
9 more
  1. 2026TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documentscited by 0
  2. 2026DocClaw: A Unified Agentic System for Intelligent Document Processingcited by 0
  3. 2026WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humanscited by 0
  4. 2026Ancient-Bench: A Comprehensive Multi-millennial, Multi-medium, and Multi-script Benchmark for Ancient Chinese Artifact Text Recognitioncited by 0
  5. 2026Closing Cost-Quality Gap in Document VLMs: Difficulty-Aware Data Curation and Quality-Adjusted Deployment Economicscited by 0
  6. 2026Collapse, Not Complexity: Failure-Conditioned Decomposition Repair for End-to-End Document Parsingcited by 0
  7. 2026DocMIDE: Learning Multi-Hop Implicit Derivation in Visually Rich Documentscited by 0
  8. 2026Xiaomi-OCR-0 Technical Reportcited by 0
  9. 2026PolyOCR-Venus: Unified OCR Foundation Models for Text-Centric Visual Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model
  2. 2026DocOCR-Eval: A Correction-Based Framework for OCR Tool Selection Without Ground Truth
  3. 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patterns
  4. 2026Structured Layout Priors for Robust Out-of-Distribution Visual Document Understanding
  5. 2026Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images
  6. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Prompting
  7. 2026PARL: Position-Aware Relation Learning Network for Document Layout Analysis
  8. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing