mlsanityData to 7 Oct 2026

Hierarchical multimodal transformers for Multi-Page DocVQA

arXiv 2212.05935 ↗First posted 7 Dec 2022cs.CV cs.AI cs.CLcited by 102

Not on the map. Why?

Cited by 36 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Document Visual Question Answering (DocVQA) refers to the task of answering questions from document images. Existing work on DocVQA only considers single-page documents. However, in real scenarios documents are mostly composed of multiple pages that should be processed altogether. In this work we extend DocVQA to the multi-page scenario. For that, we first create a new dataset, MP-DocVQA, where questions are posed over multi-page documents instead of single pages. Second, we propose a new hierarchical method, Hi-VT5, based on the T5 architecture, that overcomes the limitations of current methods to process long multi-page documents. The proposed method is based on a hierarchical transformer architecture where the encoder summarizes the most relevant information of every page and then, the decoder takes this summarized information to generate the final answer. Through extensive experimentation, we demonstrate that our method is able, in a single stage, to answer the questions and provide the page that contains the relevant information to find the answer, which can be used as a kind of explainability measure.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 7 Dec 2022.

Builds on

31 of 37 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  9. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  10. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  11. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  12. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
19 more
  1. 2021InfographicVQAcited by 430
  2. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  3. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
  4. 2021OCR-free Document Understanding Transformercited by 400
  5. 2019Scene Text Visual Question Answeringcited by 327
  6. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  7. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  8. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  9. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  10. 2020On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answeringcited by 78
  11. 2022End-to-end Document Recognition and Understanding with Dessurtcited by 73
  12. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  13. 2020LAMBERT: Layout-Aware (Language) Modeling for information extractioncited by 53
  14. 2021Document Collection Visual Question Answeringcited by 45
  15. 2022OCR-IDL: OCR Annotations for Industry Document Library Datasetcited by 35
  16. 2020Cross-modal Knowledge Reasoning for Knowledge-based Visual Question Answeringcited by 32
  17. 2020Document Visual Question Answering Challenge 2020cited by 20
  18. 2021ICDAR 2021 Competition on Document VisualQuestion Answeringcited by 19
  19. 2020Multimodal grid features and cell pointers for Scene Text Visual Question Answeringcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 30; 2025: 36; 2026: 28.

102 papers.

Sort:

By area:

  1. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  2. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  3. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  4. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  5. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  6. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  7. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
  8. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  9. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  10. 2024ScreenAI: A Vision-Language Model for UI and Infographics Understandingcited by 97
  11. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 81
  12. 2024DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMscited by 72
38 more
  1. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 70
  2. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 66
  3. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  4. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  5. 2025MMDocIR: Benchmarking Multimodal Retrieval for Long Documentscited by 43
  6. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answeringcited by 33
  7. 2024DocVLM: Make Your VLM an Efficient Readercited by 27
  8. 2025SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinementcited by 27
  9. 2025FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understandingcited by 26
  10. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  11. 2024SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understandingcited by 25
  12. 2023Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answeringcited by 24
  13. 2024GRAM: Global Reasoning for Multi-Page VQAcited by 23
  14. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  15. 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performancecited by 19
  16. 2025Docopilot: Improving Multimodal Models for Document-Level Understandingcited by 19
  17. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Taskscited by 18
  18. 2023Privacy-Aware Document Visual Question Answeringcited by 17
  19. 2025MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrievalcited by 17
  20. 2024PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Samplingcited by 14
  21. 2025Breaking the Data Barrier -- Building GUI Agents Through Task Generalizationcited by 13
  22. 2025VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understandingcited by 13
  23. 2025ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoningcited by 13
  24. 2023Beyond Document Page Classification: Design, Datasets, and Challengescited by 10
  25. 2024ANLS* -- A Universal Document Processing Metric for Generative Large Language Modelscited by 10
  26. 2024JDocQA: Japanese Document Question Answering Dataset for Generative Language Modelscited by 9
  27. 2025A Simple yet Effective Layout Token in Large Language Models for Document Understandingcited by 9
  28. 2024Multi-Page Document Visual Question Answering using Self-Attention Scoring Mechanismcited by 8
  29. 2024INST-IT: Boosting Instance Understanding via Explicit Visual Prompt Instruction Tuningcited by 8
  30. 2025BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotationscited by 8
  31. 2026Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Modelcited by 8
  32. 2023SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentationcited by 7
  33. 2024PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answeringcited by 7
  34. 2024μgat: Improving Single-Page Document Parsing by Providing Multi-Page Contextcited by 7
  35. 2024Extracting Training Data from Document-Based VQA Modelscited by 6
  36. 2025Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Ruminationcited by 6
  37. 2026MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documentscited by 6
  38. 2023Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents via Semantic-Oriented Hierarchical Graphscited by 5

50 most cited of 102 All 102 (JSON)