# PDFTriage: Question Answering over Long, Structured Documents

arXiv: https://arxiv.org/abs/2309.08872 · HTML: https://mlsanity.com/paper/2309.08872/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 16 Sep 2023.

## Builds on (most cited first, 23 of 23)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [ToolQA: A Dataset for LLM Question Answering with External Tools](https://mlsanity.com/paper/2306.13304/) (2023)
- [TALM: Tool Augmented Language Models](https://mlsanity.com/paper/2205.12255/) (2022)
- [TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs](https://mlsanity.com/paper/2303.16434/) (2023)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval](https://mlsanity.com/paper/2302.07452/) (2023)
- [Task-aware Retrieval with Instructions](https://mlsanity.com/paper/2211.09260/) (2022)
- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In](https://mlsanity.com/paper/2305.17331/) (2023)
- [Visconde: Multi-document QA with GPT-3 and Neural Reranking](https://mlsanity.com/paper/2212.09656/) (2022)
- [Synergistic Interplay between Search and Large Language Models for Information Retrieval](https://mlsanity.com/paper/2305.07402/) (2023)

## Built on by (most cited first, 29 of 29)

- [GRAG: Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2405.16506/) (2024)
- [M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework](https://mlsanity.com/paper/2411.06176/) (2024)
- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)
- [SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding](https://mlsanity.com/paper/2411.01106/) (2024)
- [Video Summarization with Large Language Models](https://mlsanity.com/paper/2504.11199/) (2025)
- [PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Sampling](https://mlsanity.com/paper/2410.05970/) (2024)
- [DocDancer: Towards Agentic Document-Grounded Information Seeking](https://mlsanity.com/paper/2601.05163/) (2026)
- [SuperRAG: Beyond RAG with Layout-Aware Graph Modeling](https://mlsanity.com/paper/2503.04790/) (2025)
- [MultiFinRAG: An Optimized Multimodal Retrieval-Augmented Generation (RAG) Framework for Financial Question Answering](https://mlsanity.com/paper/2506.20821/) (2025)
- [DocXChain: A Powerful Open-Source Toolchain for Document Parsing and Beyond](https://mlsanity.com/paper/2310.12430/) (2023)
- [HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA](https://mlsanity.com/paper/2402.01767/) (2024)
- [Dr.Academy: A Benchmark for Evaluating Questioning Capability in Education for Large Language Models](https://mlsanity.com/paper/2408.10947/) (2024)
- [MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents](https://mlsanity.com/paper/2604.12352/) (2026)
- [MoDora: Tree-Based Semi-Structured Document Analysis System](https://mlsanity.com/paper/2602.23061/) (2026)
- [Seg2Act: Global Context-aware Action Generation for Document Logical Structuring](https://mlsanity.com/paper/2410.06802/) (2024)
- [Enhancing Retrieval for ESGLLM via ESG-CID -- A Disclosure Content Index Finetuning Dataset for Mapping GRI and ESRS](https://mlsanity.com/paper/2503.10674/) (2025)
- [MinerU-Popo: Universal Post-Processing Model for Structured Document Parsing](https://mlsanity.com/paper/2605.24973/) (2026)
- [GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents](https://mlsanity.com/paper/2607.11192/) (2026)
- [MemSum-DQA: Adapting An Efficient Long Document Extractive Summarizer for Document Question Answering](https://mlsanity.com/paper/2310.06436/) (2023)
- [DistilDoc: Knowledge Distillation for Visually-Rich Document Applications](https://mlsanity.com/paper/2406.08226/) (2024)
- [GE-Chat: A Graph Enhanced RAG Framework for Evidential Response Generation of LLMs](https://mlsanity.com/paper/2505.10143/) (2025)
- [EidosDoc: Implicit Structure Encoding for Cost-Effective Semi-Structured Document QA](https://mlsanity.com/paper/2609.27784/) (2026)
- [PDF Retrieval Augmented Question Answering](https://mlsanity.com/paper/2506.18027/) (2025)
- [LARAG: Link-Aware Retrieval Strategy for RAG Systems in Hyperlinked Technical Documentation](https://mlsanity.com/paper/2605.07517/) (2026)
- [What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering](https://mlsanity.com/paper/2608.14841/) (2026)
- [LongDocBench: Benchmarking TOC Hierarchy and Contextual Relationship Recovery in Long Documents](https://mlsanity.com/paper/2608.15064/) (2026)
- [Text, Pixels, or Both? Evaluating Input Representations for Multimodal Document QA](https://mlsanity.com/paper/2609.22628/) (2026)
- [Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses](https://mlsanity.com/paper/2609.35255/) (2026)
- [RIT-RAG: Navigating Document Corpora with Retrieval-Induced Trees](https://mlsanity.com/paper/2610.11370/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have issues with document question answering (QA) in situations where the document is unable to fit in the small context length of an LLM. To overcome this issue, most existing works focus on retrieving the relevant context from the document, representing them as plain text. However, documents such as PDFs, web pages, and presentations are naturally structured with different pages, tables, sections, and so on. Representing such structured documents as plain text is incongruous with the user's mental model of these documents with rich structure. When a system has to query the document for context, this incongruity is brought to the fore, and seemingly trivial questions can trip up the QA system. To bridge this fundamental gap in handling structured documents, we propose an approach called PDFTriage that enables models to retrieve the context based on either structure or content. Our experiments demonstrate the effectiveness of the proposed PDFTriage-augmented models across several classes of questions where existing retrieval-augmented LLMs fail. To facilitate further research on this fundamental problem, we release our benchmark dataset consisting of 900+ human-generated questions over 80 structured documents from 10 different categories of question types for document QA. Our code and datasets will be released soon on Github.