mlsanityData to 7 Oct 2026

Towards Complex Document Understanding By Discrete Reasoning

arXiv 2207.11871 ↗First posted 25 Jul 2022cs.CV cs.AIcited by 82

Not on the map. Why?

Abstract (arXiv)

Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language, which is an emerging research topic for both Natural Language Processing and Computer Vision. In this work, we introduce a new Document VQA dataset, named TAT-DQA, which consists of 3,067 document pages comprising semi-structured table(s) and unstructured text as well as 16,558 question-answer pairs by extending the TAT-QA dataset. These documents are sampled from real-world financial reports and contain lots of numbers, which means discrete reasoning capability is demanded to answer questions on this dataset. Based on TAT-DQA, we further develop a novel model named MHST that takes into account the information in multi-modalities, including text, layout and visual image, to intelligently address different types of questions with corresponding strategies, i.e., extraction or reasoning. Extensive experiments show that the MHST model significantly outperforms the baseline methods, demonstrating its effectiveness. However, the performance still lags far behind that of expert humans. We expect that our new TAT-DQA dataset would facilitate the research on deep understanding of visually-rich documents combining vision and language, especially for scenarios that require discrete reasoning. Also, we hope the proposed model would inspire researchers to design more advanced Document VQA models in future. Our dataset will be publicly available for non-commercial use at https://nextplusplus.github.io/TAT-DQA/.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 39 of its 51 references that are in the map. First posted on arXiv 25 Jul 2022.

Builds on

39 of 51 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2019Towards VQA Models That Can Readcited by 1,696
  6. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  7. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  8. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  9. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  10. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  11. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  12. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
27 more
  1. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  2. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  3. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  4. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  5. 2019Scene Text Visual Question Answeringcited by 327
  6. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 254
  7. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  8. 2019Image-based table recognition: data, model, and evaluationcited by 233
  9. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  10. 2020DocBank: A Benchmark Dataset for Document Layout Analysiscited by 152
  11. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  12. 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 122
  13. 2019TableBank: A Benchmark Dataset for Table Detection and Recognitioncited by 102
  14. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  15. 2021WebSRC: A Dataset for Web-Based Structural Reading Comprehensioncited by 98
  16. 2021StructuralLM: Structural Pre-training for Form Understandingcited by 91
  17. 2021Document AI: Benchmarks, Models and Applicationscited by 61
  18. 2019NumNet: Machine Reading Comprehension with Numerical Reasoningcited by 59
  19. 2018Translating a Math Word Problem to an Expression Treecited by 57
  20. 2019Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehensioncited by 51
  21. 2018Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problemscited by 49
  22. 2019A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoningcited by 41
  23. 2020Question Directed Graph Attention Network for Numerical Reasoning over Textcited by 35
  24. 2020A Survey of Deep Learning Approaches for OCR and Document Understandingcited by 32
  25. 2020Kleister: A novel task for Information Extraction involving Long Documents with Complex Layoutcited by 27
  26. 2021LAMPRET: Layout-Aware Multimodal PreTraining for Document Understandingcited by 15
  27. 2019A Simple and Effective Model for Answering Multi-span Questionscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 6; 2024: 17; 2025: 31; 2026: 23.

82 papers.

Sort:

By area:

  1. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 190
  2. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  3. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  4. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  5. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  6. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 102
  7. 2023Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solutioncited by 54
  8. 2025FineVision: Open Data Is All You Needcited by 51
  9. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  10. 2022PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Financecited by 45
  11. 2025MMDocIR: Benchmarking Multimodal Retrieval for Long Documentscited by 43
  12. 2024UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysiscited by 40
38 more
  1. 2024jina-clip-v2: Multilingual Multimodal Embeddings for Text and Imagescited by 36
  2. 2025REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmarkcited by 34
  3. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answeringcited by 33
  4. 2024InstructDoc: A Dataset for Zero-Shot Generalization of Visual Document Understanding with Instructionscited by 27
  5. 2024DocVLM: Make Your VLM an Efficient Readercited by 27
  6. 2025Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddingscited by 26
  7. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarioscited by 22
  8. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  9. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 20
  10. 2024TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Datacited by 19
  11. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Taskscited by 18
  12. 2024MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understandingcited by 18
  13. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAGcited by 18
  14. 2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18
  15. 2022Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decodercited by 17
  16. 2024HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Modelscited by 16
  17. 2024RoDLA: Benchmarking the Robustness of Document Layout Analysis Modelscited by 12
  18. 2024WebQuest: A Benchmark for Multimodal QA on Web Page Sequencescited by 12
  19. 2023Multi-View Graph Representation Learning for Answering Hybrid Numerical Reasoning Questioncited by 10
  20. 2023Beyond Document Page Classification: Design, Datasets, and Challengescited by 10
  21. 2025Guided Query Refinement: Multimodal Hybrid Retrieval with Test-Time Optimizationcited by 10
  22. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  23. 2025Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrievalcited by 8
  24. 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Reviewcited by 8
  25. 2025LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sightcited by 8
  26. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 8
  27. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  28. 2022A Survey on Table-and-Text HybridQA: Concepts, Methods, Challenges and Future Directionscited by 7
  29. 2023RealCQA: Scientific Chart Question Answering as a Test-bed for First-Order Logiccited by 7
  30. 2025Towards Temporal-Aware Multi-Modal Retrieval Augmented Generation in Financecited by 7
  31. 2024FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Modelscited by 6
  32. 2025Visual-TableQA: Open-Domain Benchmark for Reasoning over Table Imagescited by 6
  33. 2023Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents via Semantic-Oriented Hierarchical Graphscited by 5
  34. 2026DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splittingcited by 5
  35. 2022NAPG: Non-Autoregressive Program Generation for Hybrid Tabular-Textual Question Answeringcited by 4
  36. 2025Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and Trendscited by 4
  37. 2025Multi-Agent Interactive Question Generation Framework for Long Document Understandingcited by 4
  38. 2025PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistenciescited by 4

50 most cited of 82 All 82 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering
  2. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  3. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  4. 2026TestHallVQA: Exploring LVLMs' Document-Level Reasoning under Redundant Contexts from Scientific Exams
  5. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  6. 2026Index Light, Reason Deep: Deferred Visual Ingestion for Visual-Dense Document Question Answering
  7. 2026Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
  8. 2024Generalizing Visual Question Answering from Synthetic to Human-Written Questions via a Chain of QA with a Large Language Model