mlsanityData to 7 Oct 2026

Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training

arXiv 2603.23885 ↗First posted 25 Mar 2026cs.CVcited by 7

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Document parsing has recently advanced with multimodal large language models (MLLMs) that directly map document images to structured outputs. Traditional cascaded pipelines depend on precise layout analysis and often fail under casually captured or non-standard conditions. Although end-to-end approaches mitigate this dependency, they still exhibit repetitive, hallucinated, and structurally inconsistent predictions - primarily due to the scarcity of large-scale, high-quality full-page (document-level) end-to-end parsing data and the lack of structure-aware training strategies. To address these challenges, we propose a data-training co-design framework for robust end-to-end document parsing. A Realistic Scene Synthesis strategy constructs large-scale, structurally diverse full-page end-to-end supervision by composing layout templates with rich document elements, while a Document-Aware Training Recipe introduces progressive learning and structure-token optimization to enhance structural fidelity and decoding stability. We further build Wild-OmniDocBench, a benchmark derived from real-world captured documents for robustness evaluation. Integrated into a 1B-parameter MLLM, our method achieves superior accuracy and robustness across both scanned/digital and real-world captured scenarios. All models, data synthesis pipelines, and benchmarks will be publicly released to advance future research in document understanding.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 58 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

45 of 58 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model2 Dec
  2. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model16 Oct
  3. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing26 Sep
  4. 2025POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversion1 Sep
  5. 2025WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild?16 May
  6. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models25 Feb
  7. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding8 Apr

Other references · 38

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  8. 2021OCR-free Document Understanding Transformercited by 400
  9. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  10. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 255
  11. 2019Image-based table recognition: data, model, and evaluationcited by 233
  12. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 174
26 more
  1. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 174
  2. 2020ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Networkcited by 170
  3. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  4. 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 146
  5. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 144
  6. 2025PaddleOCR 3.0 Technical Reportcited by 136
  7. 2020Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spottingcited by 114
  8. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
  9. 2025A Comprehensive Survey on Long Context Language Modelingcited by 113
  10. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  11. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
  12. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 81
  13. 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 80
  14. 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 78
  15. 2022Syntax-Aware Network for Handwritten Mathematical Expression Recognitioncited by 68
  16. 2024Focus Anywhere for Fine-grained Multi-page Document Understandingcited by 62
  17. 2023Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modelingcited by 41
  18. 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 41
  19. 2021CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognitioncited by 36
  20. 2021Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformercited by 34
  21. 2022CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognitioncited by 24
  22. 2023Context Perception Parallel Decoder for Scene Text Recognitioncited by 19
  23. 2024Instruction-Guided Scene Text Recognitioncited by 17
  24. 2023IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognitioncited by 12
  25. 2024Arbitrary Reading Order Scene Text Spotter with Local Semantics Guidancecited by 11
  26. 2024LDP: Generalizing to Multilingual Visual Information Extraction by Language Decoupled Pretrainingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild
  2. 2025DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation
  3. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
  4. 2026Structured Layout Priors for Robust Out-of-Distribution Visual Document Understanding
  5. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding
  6. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Prompting
  7. 2026Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
  8. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding