mlsanityData to 14 Oct 2026

Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and Trends

arXiv 2501.02235 ↗First posted 4 Jan 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

The field of visually-rich document understanding, which involves interacting with visually-rich documents (whether scanned or born-digital), is rapidly evolving and still lacks consensus on several key aspects of the processing pipeline. In this work, we provide a comprehensive overview of state-of-the-art approaches, emphasizing their strengths and limitations, pointing out the main challenges in the field, and proposing promising research directions.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 71 of its 97 references that are in the map. First posted on arXiv 4 Jan 2025.

Builds on

71 of 97 references.

References · 71

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  9. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  10. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  11. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  12. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
38 more
  1. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  2. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 402
  3. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
  4. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  5. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  6. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  7. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  8. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  9. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  10. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  11. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  12. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
  13. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understandingcited by 168
  14. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
  15. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
  16. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  17. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 151
  18. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  19. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 135
  20. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  21. 2021SelfDoc: Self-Supervised Document Representation Learningcited by 117
  22. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 114
  23. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 111
  24. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  25. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 105
  26. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  27. 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
  28. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
  29. 2021StructuralLM: Structural Pre-training for Form Understandingcited by 91
  30. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  31. 2023UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understandingcited by 74
  32. 2022End-to-end Document Recognition and Understanding with Dessurtcited by 73
  33. 2022ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understandingcited by 67
  34. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  35. 2024TextSquare: Scaling up Text-Centric Visual Instruction Tuningcited by 62
  36. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  37. 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extractioncited by 44
  38. 2021A Simple and Effective Positional Encoding for Transformerscited by 37

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025LLM/Agent-as-Data-Analyst: A Surveycited by 16
  2. 2026CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embeddingcited by 7
  3. 2026UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documentscited by 5
  4. 2026ViSAR: Training-Free Adaptive-k Retrieval for Visual Document Question Answeringcited by 0