mlsanityData to 14 Oct 2026

Knowledge Graph Prompting for Multi-Document Question Answering

arXiv 2308.11730 ↗First posted 22 Aug 2023cs.CL cs.AI cs.IRcited by 152

Not on the map. Why?

Cited by 78 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The `pre-train, prompt, predict' paradigm of large language models (LLMs) has achieved remarkable success in open-domain question answering (OD-QA). However, few works explore this paradigm in the scenario of multi-document question answering (MD-QA), a task demanding a thorough understanding of the logical associations among the contents and structures of different documents. To fill this crucial gap, we propose a Knowledge Graph Prompting (KGP) method to formulate the right context in prompting LLMs for MD-QA, which consists of a graph construction module and a graph traversal module. For graph construction, we create a knowledge graph (KG) over multiple documents with nodes symbolizing passages or document structures (e.g., pages/tables), and edges denoting the semantic/lexical similarity between passages or intra-document structural relations. For graph traversal, we design an LLM-based graph traversal agent that navigates across nodes and gathers supporting passages assisting LLMs in MD-QA. The constructed graph serves as the global ruler that regulates the transitional space among passages and reduces retrieval latency. Concurrently, the graph traversal agent acts as a local navigator that gathers pertinent context to progressively approach the question and guarantee retrieval quality. Extensive experiments underscore the efficacy of KGP for MD-QA, signifying the potential of leveraging graphs in enhancing the prompt design for LLMs. Our code: https://github.com/YuWVandy/KG-LLM-MDQA.

Key facts. Cited by 152 papers in the map (data through 14 Oct 2026). Builds on 42 of its 89 references that are in the map. First posted on arXiv 22 Aug 2023.

Builds on

42 of 89 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  12. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
30 more
  1. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  2. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  3. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  4. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  5. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  6. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  7. 2020Heterogeneous Graph Transformercited by 582
  8. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  9. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  10. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  11. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 388
  12. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  13. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  14. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  15. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  16. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 167
  17. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  18. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  19. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  20. 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
  21. 2019Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QAcited by 63
  22. 2019Knowledge Guided Text Retrieval and Reading for Open Domain Question Answeringcited by 59
  23. 2020IIRC: A Dataset of Incomplete Information Reading Comprehension Questionscited by 53
  24. 2023Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answeringcited by 28
  25. 2022Visconde: Multi-document QA with GPT-3 and Neural Rerankingcited by 23
  26. 2022Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answeringcited by 22
  27. 2022Modeling Multi-hop Question Answering as Single Sequence Predictioncited by 21
  28. 2021Question Answering Survey: Directions, Challenges, Datasets, Evaluation Matricescited by 17
  29. 2023A Survey of Graph Prompting Methods: Techniques, Applications, and Challengescited by 10
  30. 2018Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehensioncited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 40; 2025: 62; 2026: 44.

152 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  2. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  3. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  4. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  5. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 90
  6. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 88
  7. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 80
  8. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 66
  9. 2023Graph Neural Prompting with Large Language Modelscited by 64
  10. 2024CompAct: Compressing Retrieved Documents Actively for Question Answeringcited by 64
  11. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  12. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
38 more
  1. 2025Knowledge Graph-Guided Retrieval Augmented Generationcited by 58
  2. 2024Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Reviewcited by 53
  3. 2024Can we Soft Prompt LLMs for Graph Learning Tasks?cited by 52
  4. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 49
  5. 2025LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corporacited by 49
  6. 2023Making Large Language Models Perform Better in Knowledge Graph Completioncited by 48
  7. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 48
  8. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 46
  9. 2024GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Modelscited by 43
  10. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworkscited by 42
  11. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  12. 2025Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunitiescited by 31
  13. 2025KET-RAG: A Cost-Efficient Multi-Granular Indexing Framework for Graph-RAGcited by 29
  14. 2025ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generationcited by 28
  15. 2025NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodescited by 26
  16. 2025GraphRAG-Bench: Challenging Domain-Specific Reasoning for Evaluating Graph Retrieval-Augmented Generationcited by 24
  17. 2024Entity Alignment with Noisy Annotations from Large Language Modelscited by 22
  18. 2025Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challengescited by 19
  19. 2024Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learningcited by 18
  20. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  21. 2023AutoKG: Efficient Automated Knowledge Graph Generation for Language Modelscited by 17
  22. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  23. 2024GeAR: Graph-enhanced Agent for Retrieval-augmented Generationcited by 17
  24. 2025Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoningcited by 17
  25. 2024Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoningcited by 16
  26. 2025BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documentscited by 16
  27. 2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answeringcited by 14
  28. 2024Towards LifeSpan Cognitive Systemscited by 13
  29. 2025AssertionForge: Enhancing Formal Verification Assertion Generation with Structured Representation of Specifications and RTLcited by 13
  30. 2024EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLMcited by 12
  31. 2024Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphscited by 12
  32. 2024KG-Retriever: Efficient Knowledge Indexing for Retrieval-Augmented Large Language Modelscited by 12
  33. 2025GraphRAG-R1: Graph Retrieval-Augmented Generation with Process-Constrained Reinforcement Learningcited by 12
  34. 2026Memory in the LLM Era: Modular Architectures and Strategies in a Unified Frameworkcited by 12
  35. 2025Mixture of Structural-and-Textual Retrieval over Text-rich Graph Knowledge Basescited by 11
  36. 2024Graph Neural Network Enhanced Retrieval for Question Answering of LLMscited by 10
  37. 2025LAG: Logic-Augmented Generation from a Cartesian Perspectivecited by 10
  38. 2024Causal Reasoning in Large Language Models: A Knowledge Graph Approachcited by 9

50 most cited of 152 All 152 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Fusing Knowledge and Language: A Comparative Study of Knowledge Graph-Based Question Answering with LLMs
  2. 2025BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering
  3. 2024Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text
  4. 2024Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains
  5. 2026The Answer Path and the Grounding Instruction in LLM Question Answering over Knowledge Graphs
  6. 2026KGCaRe: Explainable Complex Conditional Question Answering using Automatic Knowledge Graph Construction and Context Retrieval with LLMs
  7. 2025DAMR: Efficient and Adaptive Context-Aware Knowledge Graph Question Answering with LLM-Guided MCTS
  8. 2026Topology-Aware Reasoning over Incomplete Knowledge Graph with Graph-Based Soft Prompting