# PDFVQA: A New Dataset for Real-World VQA on PDF Documents

arXiv: https://arxiv.org/abs/2304.06447 · HTML: https://mlsanity.com/paper/2304.06447/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [InfographicVQA](https://mlsanity.com/paper/2104.12756/) (2021)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [DVQA: Understanding Data Visualizations via Question Answering](https://mlsanity.com/paper/1801.08163/) (2018)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [PlotQA: Reasoning over Scientific Plots](https://mlsanity.com/paper/1909.00997/) (2019)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021)
- [VisualMRC: Machine Reading Comprehension on Document Images](https://mlsanity.com/paper/2101.11272/) (2021)
- [Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA](https://mlsanity.com/paper/1911.06258/) (2019)
- [On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering](https://mlsanity.com/paper/2002.10215/) (2020)
- [LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis](https://mlsanity.com/paper/2103.15348/) (2021)
- [LEAF-QA: Locate, Encode & Attend for Figure Question Answering](https://mlsanity.com/paper/1907.12861/) (2019)
- [VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relations](https://mlsanity.com/paper/2105.06220/) (2021)
- [Multimodal Pre-training Based on Graph Attention Network for Document Understanding](https://mlsanity.com/paper/2203.13530/) (2022)
- [V-Doc : Visual questions answers with Documents](https://mlsanity.com/paper/2205.13724/) (2022)
- [Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis](https://mlsanity.com/paper/2208.10970/) (2022)
- [Visual FUDGE: Form Understanding via Dynamic Graph Editing](https://mlsanity.com/paper/2105.08194/) (2021)

## Built on by (most cited first, 28 of 28)

- [NVLM: Open Frontier-Class Multimodal LLMs](https://mlsanity.com/paper/2409.11402/) (2024)
- [M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding](https://mlsanity.com/paper/2411.04952/) (2024)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [FineVision: Open Data Is All You Need](https://mlsanity.com/paper/2510.17269/) (2025)
- [UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis](https://mlsanity.com/paper/2406.15187/) (2024)
- [LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding](https://mlsanity.com/paper/2605.27365/) (2026)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024)
- [StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyond](https://mlsanity.com/paper/2405.21013/) (2024)
- [RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning](https://mlsanity.com/paper/2402.14840/) (2024)
- [LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight](https://mlsanity.com/paper/2511.20648/) (2025)
- [PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering](https://mlsanity.com/paper/2404.12720/) (2024)
- [MemSum-DQA: Adapting An Efficient Long Document Extractive Summarizer for Document Question Answering](https://mlsanity.com/paper/2310.06436/) (2023)
- [The MERIT Dataset: Modelling and Efficiently Rendering Interpretable Transcripts](https://mlsanity.com/paper/2409.00447/) (2024)
- [LiGT: Layout-infused Generative Transformer for Visual Question Answering on Vietnamese Receipts](https://mlsanity.com/paper/2502.19202/) (2025)
- [GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMs](https://mlsanity.com/paper/2510.21501/) (2025)
- [MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use](https://mlsanity.com/paper/2511.17881/) (2025)
- [Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models](https://mlsanity.com/paper/2604.00161/) (2026)
- [DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation](https://mlsanity.com/paper/2605.15532/) (2026)
- [Workshop on Document Intelligence Understanding](https://mlsanity.com/paper/2307.16369/) (2023)
- [Jaeger: A Concatenation-Based Multi-Transformer VQA Model](https://mlsanity.com/paper/2310.07091/) (2023)
- [Enhancing BERT-Based Visual Question Answering through Keyword-Driven Sentence Selection](https://mlsanity.com/paper/2310.09432/) (2023)
- [SynDoc: A Hybrid Discriminative-Generative Framework for Enhancing Synthetic Domain-Adaptive Document Key Information Extraction](https://mlsanity.com/paper/2509.23273/) (2025)
- [VERSE: Visual Embedding Reduction and Space Exploration. Clustering-Guided Insights for Training Data Enhancement in Visually-Rich Document Understanding](https://mlsanity.com/paper/2601.05125/) (2026)
- [RobustVisRAG: Causality-Aware Vision-Based Retrieval-Augmented Generation under Visual Degradations](https://mlsanity.com/paper/2602.22013/) (2026)
- [From Content to Audience: A Multimodal Annotation Framework for Broadcast Television Analytics](https://mlsanity.com/paper/2603.26772/) (2026)
- [Xuanwu: Evolving General Multimodal Models into an Industrial-Grade Foundation for Content Ecosystems](https://mlsanity.com/paper/2603.29211/) (2026)
- [Lightweight and Production-Ready PDF Visual Element Parsing](https://mlsanity.com/paper/2604.23276/) (2026)

## Abstract (arXiv)

Document-based Visual Question Answering examines the document understanding of document images in conditions of natural language questions. We proposed a new document-based VQA dataset, PDF-VQA, to comprehensively examine the document understanding from various aspects, including document element recognition, document layout structural understanding as well as contextual understanding and key information extraction. Our PDF-VQA dataset extends the current scale of document understanding that limits on the single document page to the new scale that asks questions over the full document of multiple pages. We also propose a new graph-based VQA model that explicitly integrates the spatial and hierarchically structural relationships between different document elements to boost the document structural understanding. The performances are compared with several baselines over different question types and tasksfootnoteThe full dataset will be released after paper acceptance.