mlsanityData to 14 Oct 2026

WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild?

arXiv 2505.11015 ↗First posted 16 May 2025cs.CVcited by 26

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

The rapid advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced capabilities in Document Understanding. However, prevailing benchmarks like DocVQA and ChartQA predominantly comprise textitscanned or digital documents, inadequately reflecting the intricate challenges posed by diverse real-world scenarios, such as variable illumination and physical distortions. This paper introduces WildDoc, the inaugural benchmark designed specifically for assessing document understanding in natural environments. WildDoc incorporates a diverse set of manually captured document images reflecting real-world conditions and leverages document sources from established benchmarks to facilitate comprehensive comparisons with digital or scanned documents. Further, to rigorously evaluate model robustness, each document is captured four times under different conditions. Evaluations of state-of-the-art MLLMs on WildDoc expose substantial performance declines and underscore the models' inadequate robustness compared to traditional benchmarks, highlighting the unique challenges posed by real-world document understanding. Our project homepage is available at https://bytedance.github.io/WildDoc.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 49 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

32 of 49 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning31 Dec
  2. 2024HRVDA: High-Resolution Visual Document Assistant10 Apr
  3. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document7 Mar
  4. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding20 Nov
  5. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding4 Jul

Other references · 27

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  7. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  8. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  9. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  10. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  11. 2016A Diagram Is Worth A Dozen Imagescited by 804
  12. 2025Seed1.5-VL Technical Reportcited by 373
15 more
  1. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  2. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 280
  3. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  4. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
  5. 2024TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergycited by 71
  6. 2024TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domainscited by 69
  7. 2024TextSquare: Scaling up Text-Centric Visual Instruction Tuningcited by 62
  8. 2024A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understandingcited by 57
  9. 2025Scalable Vision Language Model Training via High Quality Data Curationcited by 55
  10. 2024Harmonizing Visual Text Comprehension and Generationcited by 54
  11. 2023Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizercited by 53
  12. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 42
  13. 2024ParGo: Bridging Vision-Language with Partial and Global Viewscited by 40
  14. 2024Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramidcited by 26
  15. 2024MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmarkcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 25.

26 papers.

Sort:

By area:

  1. 2026Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wildcited by 12
  2. 2025DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translationcited by 9
  3. 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Trainingcited by 7
  4. 2026ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formatscited by 2
  5. 2026StrucTab: A Structured Optimization Framework for Table Parsingcited by 2
  6. 2026LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Datacited by 0
  7. 2026Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Methodcited by 0
  8. 2026MA-LipNet: Multi-Dimensional Attention Networks for Robust Lipreadingcited by 0
  9. 2026The Diffusion Duet: Harmonizing Dual Channels with Wavelet Suppression for Image Separationcited by 0
  10. 2026Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignmentcited by 0
  11. 2026Efficient Image Super-Resolution with Multi-Scale Spatial Adaptive Attention Networkscited by 0
  12. 2026A Deformable Attention-Based Detection Transformer with Cross-Scale Feature Fusion for Industrial Coil Spring Inspectioncited by 0
14 more
  1. 2026Transmittance-Guided Structure-Texture Decomposition for Nighttime Image Dehazingcited by 0
  2. 2026Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Predictioncited by 0
  3. 2026Adaptive Slicing-Assisted Hyper Inference for Enhanced Small Object Detection in High-Resolution Imagerycited by 0
  4. 2026Feature Perturbation Pool-based Fusion Network for Unified Multi-Class Industrial Defect Detectioncited by 0
  5. 2026ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstructioncited by 0
  6. 2026Lightweight Real-Time Rendering Parameter Optimization via XGBoost-Driven Lookup Tablescited by 0
  7. 2026Image Classification via Random Dilated Convolution with Multi-Branch Feature Extraction and Context Excitationcited by 0
  8. 2026Gait Recognition via Deep Residual Networks and Multi-Branch Feature Fusioncited by 0
  9. 2026Local Spatiotemporal Convolutional Network for Robust Gait Recognitioncited by 0
  10. 2026Do You Need Text Rectification? Soft Attention Mask Embedding for Rectification-Free Scene Text Spottingcited by 0
  11. 2026DAIN: Dynamic Agent-Based Interaction Network for Efficient and Collaborative Multimodal Reasoningcited by 0
  12. 2026ProWAFT: A ROMA-LPD Instance for Workload-Aware and Dynamic Fault Tolerance in FPGA-Based CNN Acceleratorscited by 0
  13. 2026Rank-Consistent Set Reasoning for Co-Salient Object Detectioncited by 0
  14. 2026Contour-Guided Spectral Routing for Robust Real-Time Pedestrian Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humans
  2. 2025MosaicDoc: A Large-Scale Bilingual Benchmark for Visually Rich Document Understanding
  3. 2024MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding
  4. 2026SynthDocBench: Controlled Benchmark for Long-Context Visual Document Understanding
  5. 2026CC-OCR V2: Fine-Grained Attribution of LMM Failures in Real-World Visual Document Understanding
  6. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating
  7. 2025Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark
  8. 2024DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models