# Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training arXiv: https://arxiv.org/abs/2603.23885 ยท HTML: https://mlsanity.com/paper/2603.23885/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 45 of its 58 references that are in the map. - First posted on arXiv 25 Mar 2026. ## Placed in - Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/) ## Builds on (most cited first, 30 of 45) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023) - [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025) - [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025) - [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021) - [Effective Long-Context Scaling of Foundation Models](https://mlsanity.com/paper/2309.16039/) (2023) - [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024) - [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019) - [DeepSeek-OCR: Contexts Optical Compression](https://mlsanity.com/paper/2510.18234/) (2025) - [General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model](https://mlsanity.com/paper/2409.01704/) (2024) - [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020) - [Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models](https://mlsanity.com/paper/2402.14207/) (2024) - [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024) - [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020) - [PaddleOCR 3.0 Technical Report](https://mlsanity.com/paper/2507.05595/) (2025) - [olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models](https://mlsanity.com/paper/2502.18443/) (2025) - [A Comprehensive Survey on Long Context Language Modeling](https://mlsanity.com/paper/2503.17407/) (2025) - [MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing](https://mlsanity.com/paper/2509.22186/) (2025) - [Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting](https://mlsanity.com/paper/2007.09482/) (2020) - [mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding](https://mlsanity.com/paper/2409.03420/) (2024) - [M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding](https://mlsanity.com/paper/2411.04952/) (2024) - [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024) - [PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model](https://mlsanity.com/paper/2510.14528/) (2025) - [MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm](https://mlsanity.com/paper/2506.05218/) (2025) - [Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models](https://mlsanity.com/paper/2312.06109/) (2023) - [DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception](https://mlsanity.com/paper/2410.12628/) (2024) - [Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting](https://mlsanity.com/paper/2505.14059/) (2025) ## Built on by (most cited first, 7 of 7) - [Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters](https://mlsanity.com/paper/2605.11960/) (2026) - [ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats](https://mlsanity.com/paper/2606.01348/) (2026) - [StrucTab: A Structured Optimization Framework for Table Parsing](https://mlsanity.com/paper/2606.29905/) (2026) - [MORE: A Multilingual Document Parsing Benchmark and Evaluation](https://mlsanity.com/paper/2607.02956/) (2026) - [TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents](https://mlsanity.com/paper/2608.12898/) (2026) - [WeVisDoc: From Coverage to Capability for Robust End-to-End Document Parsing](https://mlsanity.com/paper/2609.20423/) (2026) - [Xiaomi-OCR-0 Technical Report](https://mlsanity.com/paper/2609.36136/) (2026) ## Abstract (arXiv) Document parsing has recently advanced with multimodal large language models (MLLMs) that directly map document images to structured outputs. Traditional cascaded pipelines depend on precise layout analysis and often fail under casually captured or non-standard conditions. Although end-to-end approaches mitigate this dependency, they still exhibit repetitive, hallucinated, and structurally inconsistent predictions - primarily due to the scarcity of large-scale, high-quality full-page (document-level) end-to-end parsing data and the lack of structure-aware training strategies. To address these challenges, we propose a data-training co-design framework for robust end-to-end document parsing. A Realistic Scene Synthesis strategy constructs large-scale, structurally diverse full-page end-to-end supervision by composing layout templates with rich document elements, while a Document-Aware Training Recipe introduces progressive learning and structure-token optimization to enhance structural fidelity and decoding stability. We further build Wild-OmniDocBench, a benchmark derived from real-world captured documents for robustness evaluation. Integrated into a 1B-parameter MLLM, our method achieves superior accuracy and robustness across both scanned/digital and real-world captured scenarios. All models, data synthesis pipelines, and benchmarks will be publicly released to advance future research in document understanding.