How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings
Not on the map. Why?
Abstract (arXiv)
Document parsing models are improving rapidly, but existing benchmarks are still insufficient for a comprehensive evaluation of their capabilities. We introduce PureDocBench, a benchmark with 10 domains, 66 subcategories, and 1,475 pages. Each page has matched Clean, Digital, and Real versions, covering clean renders, simulated degradation, and real capture or sharing conditions. The resulting 4,425 images have source-derived annotations for text, formulas, tables, and reading order, checked through automated validation and human review. Keeping page content and annotations fixed allows direct comparisons across image conditions. We evaluate 58 models, including pipeline-based parsers, end-to-end parsers, and general vision-language models. The best three-track average score is 79.69 out of 100, and the mean across models is 63.84. Scores drop by 3.34 points on Digital and 11.61 points on Real on average, relative to Clean. TeleOCR leads on Clean, GLM-5.3-Flash on Digital, and Gemini 3.6 Flash on Real. The benchmark dataset, source documents, generation tools, and evaluation code are publicly available at https://github.com/zhihengli-casia/PureDocBench.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 38 of its 75 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
38 of 75 references.
References · 38
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
- 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
- 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 256
- 2019Image-based table recognition: data, model, and evaluationcited by 237
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
- 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
- 2022DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysiscited by 148
- 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 147
- 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 126
- 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 125
- 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 116
26 more
- 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
- 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
- 2021Document AI: Benchmarks, Models and Applicationscited by 62
- 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 57
- 2025olmOCR 2: Unit Test Rewards for Document OCRcited by 47
- 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 47
- 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
- 2026STEP3-VL-10B Technical Reportcited by 32
- 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 27
- 2026GLM-OCR Technical Reportcited by 26
- 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Promptingcited by 22
- 2026Qianfan-OCR: A Unified End-to-End Model for Document Intelligencecited by 21
- 2025HunyuanOCR Technical Reportcited by 20
- 2026Multimodal OCR: Parse Anything from Documentscited by 14
- 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
- 2026FireRed-OCR Technical Reportcited by 13
- 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Modelscited by 12
- 2026Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wildcited by 12
- 2026ParseBench: A Document Parsing Benchmark for AI Agentscited by 11
- 2026Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decodingcited by 10
- 2025DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translationcited by 9
- 2025Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCRcited by 9
- 2026Logics-Parsing-Omni Technical Reportcited by 5
- 2026MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarioscited by 5
- 2026CC-OCR V2: Fine-Grained Attribution of LMM Failures in Real-World Visual Document Understandingcited by 3
- 2026Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collectionscited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026OvisOCR2 Technical Reportcited by 5
- 2026TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documentscited by 0
- 2026WeVisDoc: From Coverage to Capability for Robust End-to-End Document Parsingcited by 0
- 2026Collapse, Not Complexity: Failure-Conditioned Decomposition Repair for End-to-End Document Parsingcited by 0
- 2026Collapse, Not Complexity: Failure-Conditioned Decomposition Repair for End-to-End Document Parsing20 Sep 2026
- 2026WeVisDoc: From Coverage to Capability for Robust End-to-End Document Parsing17 Sep 2026
- 2026TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents13 Aug 2026
- 2026OvisOCR2 Technical Report15 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations
- 2026Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
- 2026From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsing
- 2026RealDocBench: A Benchmark for Field-Level QA and Layout Understanding on Real-World Regulated Documents
- 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training
- 2025DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLM
- 2026SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding
- 2026ParseFixer: An Agentic Framework for Document Parsing via Selective Multimodal Correction