mlsanityData to 7 Oct 2026

Document Understanding Dataset and Evaluation (DUDE)

arXiv 2305.08455 ↗First posted 15 May 2023cs.CV cs.CL cs.LGcited by 132

Not on the map. Why?

Cited by 40 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We call on the Document AI (DocAI) community to reevaluate current methodologies and embrace the challenge of creating more practically-oriented benchmarks. Document Understanding Dataset and Evaluation (DUDE) seeks to remediate the halted research progress in understanding visually-rich documents (VRDs). We present a new dataset with novelties related to types of questions, answers, and document layouts based on multi-industry, multi-domain, and multi-page VRDs of various origins, and dates. Moreover, we are pushing the boundaries of current methods by creating multi-task and multi-domain evaluation setups that more accurately simulate real-world situations where powerful generalization and adaptation under low-resource settings are desired. DUDE aims to set a new standard as a more practical, long-standing benchmark for the community, and we hope that it will lead to future extensions and contributions that address real-world challenges. Finally, our work illustrates the importance of finding more efficient ways to model language, images, and layout in DocAI.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). Builds on 78 of its 117 references that are in the map. First posted on arXiv 15 May 2023.

Builds on

78 of 117 references.

References · 78

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2015VQA: Visual Question Answeringcited by 2,489
  8. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  10. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  12. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
38 more
  1. 2017Selective Classification for Deep Neural Networkscited by 799
  2. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  3. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  4. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  5. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  6. 2019Measuring Calibration in Deep Learningcited by 443
  7. 2021InfographicVQAcited by 430
  8. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  9. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  10. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  11. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  12. 2019Scene Text Visual Question Answeringcited by 327
  13. 2019Verified Uncertainty Calibrationcited by 307
  14. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  15. 2019Image-based table recognition: data, model, and evaluationcited by 233
  16. 2020Selective Question Answering under Domain Shiftcited by 228
  17. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  18. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  19. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
  20. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  21. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  22. 2019Evaluating model calibration in classificationcited by 165
  23. 2020DocBank: A Benchmark Dataset for Document Layout Analysiscited by 152
  24. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  25. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 151
  26. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  27. 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 146
  28. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  29. 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 122
  30. 2021SelfDoc: Self-Supervised Document Representation Learningcited by 117
  31. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Taskscited by 110
  32. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 102
  33. 2019Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoningcited by 100
  34. 2020Mitigating Bias in Calibration Error Estimationcited by 100
  35. 2021Question Answering Over Temporal Knowledge Graphscited by 99
  36. 2022Unified Pretraining Framework for Document Understandingcited by 85
  37. 2021Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layoutscited by 82
  38. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 35; 2025: 42; 2026: 51.

132 papers.

Sort:

By area:

  1. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  2. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  3. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  4. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  5. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  6. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  7. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
  8. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  9. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  10. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 70
  11. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  12. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 57
38 more
  1. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 47
  2. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 46
  3. 2025MMDocIR: Benchmarking Multimodal Retrieval for Long Documentscited by 43
  4. 2025VisRL: Intention-Driven Visual Perception via Reinforced Reasoningcited by 39
  5. 2024DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Modelscited by 34
  6. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answeringcited by 33
  7. 2024Overcoming Common Flaws in the Evaluation of Selective Classification Systemscited by 31
  8. 2023PDFTriage: Question Answering over Long, Structured Documentscited by 29
  9. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  10. 2024DocVLM: Make Your VLM an Efficient Readercited by 27
  11. 2025SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinementcited by 27
  12. 2025Introducing Visual Perception Token into Multimodal Large Language Modelcited by 26
  13. 2024SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understandingcited by 25
  14. 2024OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generationcited by 24
  15. 2024GRAM: Global Reasoning for Multi-Page VQAcited by 23
  16. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarioscited by 22
  17. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  18. 2025Docopilot: Improving Multimodal Models for Document-Level Understandingcited by 19
  19. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Taskscited by 18
  20. 2024MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understandingcited by 18
  21. 2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18
  22. 2023Privacy-Aware Document Visual Question Answeringcited by 17
  23. 2025Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queriescited by 16
  24. 2024PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Samplingcited by 14
  25. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  26. 2025VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understandingcited by 13
  27. 2024WebQuest: A Benchmark for Multimodal QA on Web Page Sequencescited by 12
  28. 2025Handwritten Text Recognition: A Surveycited by 12
  29. 2024StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyondcited by 11
  30. 2026DocDancer: Towards Agentic Document-Grounded Information Seekingcited by 11
  31. 2023Beyond Document Page Classification: Design, Datasets, and Challengescited by 10
  32. 2024ANLS* -- A Universal Document Processing Metric for Generative Large Language Modelscited by 10
  33. 2024Bridging the Gap Between End-to-End and Two-Step Text Spottingcited by 9
  34. 2025A Simple yet Effective Layout Token in Large Language Models for Document Understandingcited by 9
  35. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  36. 2024BuDDIE: A Business Document Dataset for Multi-task Information Extractioncited by 8
  37. 2025BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotationscited by 8
  38. 2025WikiMixQA: A Multimodal Benchmark for Question Answering over Tables and Chartscited by 8

50 most cited of 132 All 132 (JSON)