# Qianfan-OCR: A Unified End-to-End Model for Document Intelligence

arXiv: https://arxiv.org/abs/2603.13398 · HTML: https://mlsanity.com/paper/2603.13398/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 22 of its 24 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 22 of 22)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models](https://mlsanity.com/paper/2305.07895/) (2023)
- [VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models](https://mlsanity.com/paper/2407.11691/) (2024)
- [Nougat: Neural Optical Understanding for Academic Documents](https://mlsanity.com/paper/2308.13418/) (2023)
- [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://mlsanity.com/paper/2406.18521/) (2024)
- [DeepSeek-OCR: Contexts Optical Compression](https://mlsanity.com/paper/2510.18234/) (2025)
- [General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model](https://mlsanity.com/paper/2409.01704/) (2024)
- [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024)
- [PaddleOCR 3.0 Technical Report](https://mlsanity.com/paper/2507.05595/) (2025)
- [olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models](https://mlsanity.com/paper/2502.18443/) (2025)
- [ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering](https://mlsanity.com/paper/2504.05506/) (2025)
- [PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsing](https://mlsanity.com/paper/2601.21957/) (2026)
- [BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks](https://mlsanity.com/paper/2412.04626/) (2024)

## Built on by (most cited first, 21 of 21)

- [HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better](https://mlsanity.com/paper/2607.04884/) (2026)
- [Unlimited OCR Works](https://mlsanity.com/paper/2606.23050/) (2026)
- [OvisOCR2 Technical Report](https://mlsanity.com/paper/2607.13639/) (2026)
- [How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings](https://mlsanity.com/paper/2605.07492/) (2026)
- [TexOCR: Advancing Document OCR Models for Compilable Page-to-LaTeX Reconstruction](https://mlsanity.com/paper/2604.22880/) (2026)
- [MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing](https://mlsanity.com/paper/2605.22100/) (2026)
- [LayoutLite: Token-Level Implicit Layout Analysis for Efficient Document OCR](https://mlsanity.com/paper/2607.22200/) (2026)
- [When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documents](https://mlsanity.com/paper/2607.24077/) (2026)
- [PaDoc: Layout-Grounded Parallel Decoding for Document Parsing](https://mlsanity.com/paper/2608.06146/) (2026)
- [All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts](https://mlsanity.com/paper/2609.24058/) (2026)
- [Edges Before Embeddings: A Confidence-Aware Blur Gate for Vision-Language Pipelines](https://mlsanity.com/paper/2606.25838/) (2026)
- [HPD-Parsing: Hierarchical Parallel Document Parsing](https://mlsanity.com/paper/2607.18839/) (2026)
- [TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents](https://mlsanity.com/paper/2608.12898/) (2026)
- [DocClaw: A Unified Agentic System for Intelligent Document Processing](https://mlsanity.com/paper/2608.18685/) (2026)
- [WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humans](https://mlsanity.com/paper/2608.22959/) (2026)
- [Ancient-Bench: A Comprehensive Multi-millennial, Multi-medium, and Multi-script Benchmark for Ancient Chinese Artifact Text Recognition](https://mlsanity.com/paper/2608.27169/) (2026)
- [Closing Cost-Quality Gap in Document VLMs: Difficulty-Aware Data Curation and Quality-Adjusted Deployment Economics](https://mlsanity.com/paper/2609.01575/) (2026)
- [Collapse, Not Complexity: Failure-Conditioned Decomposition Repair for End-to-End Document Parsing](https://mlsanity.com/paper/2609.23592/) (2026)
- [DocMIDE: Learning Multi-Hop Implicit Derivation in Visually Rich Documents](https://mlsanity.com/paper/2609.24092/) (2026)
- [Xiaomi-OCR-0 Technical Report](https://mlsanity.com/paper/2609.36136/) (2026)
- [PolyOCR-Venus: Unified OCR Foundation Models for Text-Centric Visual Intelligence](https://mlsanity.com/paper/2609.37712/) (2026)

## Abstract (arXiv)

We present Qianfan-OCR, a 4B-parameter end-to-end vision-language model that unifies document parsing, layout analysis, and document understanding within a single architecture. It performs direct image-to-Markdown conversion and supports diverse prompt-driven tasks including table extraction, chart understanding, document QA, and key information extraction. To address the loss of explicit layout analysis in end-to-end OCR, we propose Layout-as-Thought, an optional thinking phase triggered by special think tokens that generates structured layout representations -- bounding boxes, element types, and reading order -- before producing final outputs, recovering layout grounding capabilities while improving accuracy on complex layouts. Qianfan-OCR ranks first among end-to-end models on OmniDocBench v1.5 (93.12) and OlmOCR Bench (79.8), achieves competitive results on OCRBench, CCOCR, DocVQA, and ChartQA against general VLMs of comparable scale, and attains the highest average score on public key information extraction benchmarks, surpassing Gemini-3.1-Pro, Seed-2.0, and Qwen3-VL-235B. The model is publicly accessible via the Baidu AI Cloud Qianfan platform.