Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild
On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
Abstract (arXiv)
While Vision-Language Models (VLMs) achieve near-perfect scores on digital document benchmarks like OmniDocBench, their performance in the unpredictable physical world remains largely unknown due to the lack of controlled yet realistic evaluations. We introduce Real5-OmniDocBench, the first benchmark that performs a full-scale, one-to-one physical reconstruction of the entire OmniDocBench v1.5 (1,355 images) across five critical real-world scenarios: Scanning, Warping, Screen-Photography, Illumination, and Skew. Unlike prior benchmark that either lack digital correspondence or employ partial sampling, our complete ground-truth mapping enables, for the first time, rigorous factor-wise attribution of performance degradation-allowing us to pinpoint whether failures stem from geometric distortions, optical artifacts, or model limitations. Our benchmark establishes a challenging new standard for the community, demonstrating that the 'reality gap' in document parsing is far from closed, and provides a diagnostic tool to guide the development of truly resilient document intelligence.
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 19 references that are in the map. First posted on arXiv 4 Mar 2026.
Builds on
12 of 19 references.
Closest ancestors · 6
Same direction, newest first
- 2025DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation23 Nov
- 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model16 Oct
- 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing26 Sep
- 2025WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild?16 May
- 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models25 Feb
- 2025Ocean-OCR: Towards General OCR Application via a Vision-Language Model26 Jan
Other references · 6
- 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 254
- 2019Image-based table recognition: data, model, and evaluationcited by 233
- 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 81
- 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 54
- 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 39
- 2024DocRes: A Generalist Model Toward Unifying Document Image Restoration Taskscited by 27
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Trainingcited by 20
- 2026MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarioscited by 4
- 2026RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wildcited by 2
- 2026GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documentscited by 2
- 2026CC-OCR V2: Fine-Grained Attribution of LMM Failures in Real-World Visual Document Understandingcited by 1
- 2026Parser-Oriented Structural Refinement for a Stable Layout Interface in Document Parsingcited by 0
- 2026How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligencecited by 0
- 2026MUDIDI: A Two-Stage Framework for Multilingual Dictionary Digitization with Language Modelscited by 0
- 2026From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsingcited by 0
- 2026WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humanscited by 0
- 2026Xiaomi-OCR-0 Technical Reportcited by 0
- 2026Xiaomi-OCR-0 Technical Report28 Sep 2026
- 2026WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humans24 Aug 2026
- 2026From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsing10 Aug 2026
- 2026GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents13 Jul 2026
- 2026RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wild22 Jun 2026
- 2026MUDIDI: A Two-Stage Framework for Multilingual Dictionary Digitization with Language Models8 Jun 2026
- 2026PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training2 Jun 2026
- 2026How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence19 May 2026
- 2026CC-OCR V2: Fine-Grained Attribution of LMM Failures in Real-World Visual Document Understanding5 May 2026
- 2026Parser-Oriented Structural Refinement for a Stable Layout Interface in Document Parsing3 Apr 2026
- 2026MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios30 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training
- 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations
- 2026ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction
- 2026Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
- 2026Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models
- 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding
- 2026FastOCR: Dynamic Visual Fixation via KV Cache Pruning for Efficient Document Parsing
- 2026Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction