mlsanityData to 14 Oct 2026

OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations

arXiv 2412.07626 ↗First posted 10 Dec 2024cs.CV cs.AI cs.IRcited by 150

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Document content extraction is a critical task in computer vision, underpinning the data needs of large language models (LLMs) and retrieval-augmented generation (RAG) systems. Despite recent progress, current document parsing methods have not been fairly and comprehensively evaluated due to the narrow coverage of document types and the simplified, unrealistic evaluation procedures in existing benchmarks. To address these gaps, we introduce OmniDocBench, a novel benchmark featuring high-quality annotations across nine document sources, including academic papers, textbooks, and more challenging cases such as handwritten notes and densely typeset newspapers. OmniDocBench supports flexible, multi-level evaluations--ranging from an end-to-end assessment to the task-specific and attribute--based analysis using 19 layout categories and 15 attribute labels. We conduct a thorough evaluation of both pipeline-based methods and end-to-end vision-language models, revealing their strengths and weaknesses across different document types. OmniDocBench sets a new standard for the fair, diverse, and fine-grained evaluation in document parsing. Dataset and code are available at https://github.com/opendatalab/OmniDocBench.

Key facts. Cited by 150 papers in the map (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 43 of its 57 references that are in the map. First posted on arXiv 10 Dec 2024.

Builds on

43 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction28 Oct

Other references · 42

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  9. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  10. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  11. 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
  12. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
30 more
  1. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  2. 2019Image-based table recognition: data, model, and evaluationcited by 237
  3. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  4. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  5. 2020ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Networkcited by 170
  6. 2020DocBank: A Benchmark Dataset for Document Layout Analysiscited by 153
  7. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  8. 2022DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysiscited by 148
  9. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 128
  10. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 114
  11. 2022PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR Systemcited by 106
  12. 2019TableBank: A Benchmark Dataset for Table Detection and Recognitioncited by 102
  13. 2022Unified Pretraining Framework for Document Understandingcited by 85
  14. 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 82
  15. 2016Image-to-Markup Generation with Coarse-to-Fine Attentioncited by 79
  16. 2022SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognitioncited by 70
  17. 2020Spatial Dependency Parsing for Semi-Structured Document Information Extractioncited by 65
  18. 2024Focus Anywhere for Fine-grained Multi-page Document Understandingcited by 62
  19. 2021PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Networkcited by 49
  20. 2023M6Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysiscited by 46
  21. 2023Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modelingcited by 41
  22. 2018Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognitioncited by 36
  23. 2024DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Modelscited by 36
  24. 2020Improving Attention-Based Handwritten Mathematical Expression Recognition with Scale Augmentation and Drop Attentioncited by 22
  25. 2020Towards a Multi-modal, Multi-task Learning based Pre-training Framework for Document Representation Learningcited by 21
  26. 2021TabLeX: A Benchmark Dataset for Structure and Content Information Extraction from Scientific Tablescited by 15
  27. 2024READoc: A Unified Benchmark for Realistic Document Structured Extractioncited by 15
  28. 2023SwinDocSegmenter: An End-to-End Unified Domain Adaptive Transformer for Document Instance Segmentationcited by 13
  29. 2023DocXChain: A Powerful Open-Source Toolchain for Document Parsing and Beyondcited by 8
  30. 2024GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 29; 2026: 120.

150 papers.

Sort:

By area:

  1. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  2. 2026Gemma 4 Technical Reportcited by 259
  3. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  4. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  5. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
  6. 2025PaddleOCR 3.0 Technical Reportcited by 143
  7. 2026Kimi K3: Open Frontier Intelligencecited by 138
  8. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
  9. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?cited by 76
  10. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
  11. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 57
  12. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
38 more
  1. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 47
  2. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  3. 2026OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoningcited by 39
  4. 2025POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversioncited by 34
  5. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  6. 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 27
  7. 2025OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoningcited by 25
  8. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Promptingcited by 22
  9. 2026Qianfan-OCR: A Unified End-to-End Model for Document Intelligencecited by 21
  10. 2026PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Trainingcited by 21
  11. 2025Logics-Parsing Technical Reportcited by 17
  12. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsingcited by 17
  13. 2026Multimodal OCR: Parse Anything from Documentscited by 14
  14. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  15. 2025Prompt Injection Attacks on LLM Generated Reviews of Scientific Publicationscited by 13
  16. 2025UNO-Bench: A Unified Benchmark for Exploring the Compositional Law Between Uni-modal and Omni-modal in Omni Modelscited by 13
  17. 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patternscited by 13
  18. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
  19. 2026FireRed-OCR Technical Reportcited by 13
  20. 2026ParseBench: A Document Parsing Benchmark for AI Agentscited by 11
  21. 2025DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Modelcited by 10
  22. 2026Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decodingcited by 10
  23. 2025DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translationcited by 9
  24. 2025Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCRcited by 9
  25. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 9
  26. 2026Unlimited OCR Workscited by 8
  27. 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Trainingcited by 7
  28. 2025SO-Bench: A Structural Output Evaluation of Multimodal LLMscited by 6
  29. 2025DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLMcited by 6
  30. 2025SCORE: A Semantic Evaluation Framework for Generative Document Parsingcited by 5
  31. 2026DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splittingcited by 5
  32. 2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
  33. 2026MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarioscited by 5
  34. 2026Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characterscited by 5
  35. 2026CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligencecited by 5
  36. 2026Kwai Keye-VL-2.0 Technical Reportcited by 5
  37. 2026OvisOCR2 Technical Reportcited by 5
  38. 2026MOSS-VL Technical Reportcited by 5

50 most cited of 150 All 150 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings
  2. 2026Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild
  3. 2026Lightweight and Production-Ready PDF Visual Element Parsing
  4. 2026DocOCR-Eval: A Correction-Based Framework for OCR Tool Selection Without Ground Truth
  5. 2026NovaLAD: A Fast, CPU-Optimized Document Extraction Pipeline for Generative AI and Data Intelligence
  6. 2026OCRTurk: A Comprehensive OCR Benchmark for Turkish
  7. 2026Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction
  8. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding