mlsanityData to 7 Oct 2026

GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models

arXiv 2406.14550 ↗First posted 20 Jun 2024cs.CL cs.AIcited by 42

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Long-context capabilities are essential for large language models (LLMs) to tackle complex and long-input tasks. Despite numerous efforts made to optimize LLMs for long contexts, challenges persist in robustly processing long inputs. In this paper, we introduce GraphReader, a graph-based agent system designed to handle long texts by structuring them into a graph and employing an agent to explore this graph autonomously. Upon receiving a question, the agent first undertakes a step-by-step analysis and devises a rational plan. It then invokes a set of predefined functions to read node content and neighbors, facilitating a coarse-to-fine exploration of the graph. Throughout the exploration, the agent continuously records new insights and reflects on current circumstances to optimize the process until it has gathered sufficient information to generate an answer. Experimental results on the LV-Eval dataset reveal that GraphReader, using a 4k context window, consistently outperforms GPT-4-128k across context lengths from 16k to 256k by a large margin. Additionally, our approach demonstrates superior performance on four challenging single-hop and multi-hop benchmarks.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 51 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

37 of 51 references.

References · 37

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  9. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  11. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  12. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
25 more
  1. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  2. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  3. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  4. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
  5. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
  6. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 247
  7. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
  8. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 183
  9. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 150
  10. 2024A Human-Inspired Reading Agent with Gist Memory of Very Long Contextscited by 140
  11. 2023Walking Down the Memory Maze: Beyond Context Limit through Interactive Readingcited by 129
  12. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  13. 2015Yara Parser: A Fast and Accurate Dependency Parsercited by 110
  14. 2023PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Trainingcited by 109
  15. 2020Entities as Experts: Sparse Memory Access with Entity Supervisioncited by 100
  16. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  17. 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 80
  18. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  19. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  20. 2021Mention Memory: incorporating textual knowledge into Transformers through entity mention attentioncited by 42
  21. 2020Large-Scale Object Detection in the Wild from Imbalanced Multi-Labelscited by 35
  22. 2021GAIA: A Transfer Learning System of Object Detection that Fits Your Needscited by 27
  23. 2024Long Context Alignment with Short Instructions and Synthesized Positionscited by 22
  24. 2024Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Levelcited by 22
  25. 2022Visual Encoding and Debiasing for CTR Predictioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 28; 2026: 11.

42 papers.

Sort:

By area:

  1. 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 372
  2. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 100
  3. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  4. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
  5. 2025Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?cited by 43
  6. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 39
  7. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 35
  8. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  9. 2025CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehensioncited by 23
  10. 2025LongRoPE2: Near-Lossless LLM Context Window Scalingcited by 21
  11. 2024GeAR: Graph-enhanced Agent for Retrieval-augmented Generationcited by 17
  12. 2025MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environmentscited by 13
30 more
  1. 2025Adaptive Fault Tolerance Mechanisms of Large Language Models in Cloud Computing Environmentscited by 12
  2. 2025GraphRAG-R1: Graph Retrieval-Augmented Generation with Process-Constrained Reinforcement Learningcited by 12
  3. 2025BioMaze: Benchmarking and Enhancing Large Language Models for Biological Pathway Reasoningcited by 10
  4. 2025SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compressioncited by 9
  5. 2025Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaborationcited by 7
  6. 2024Scaling Knowledge Graph Construction through Synthetic Data Generation and Distillationcited by 6
  7. 2025DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Modelscited by 6
  8. 2025AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Constructioncited by 6
  9. 2025Wikontic: Constructing Wikidata-Aligned, Ontology-Aware Knowledge Graphs with Large Language Modelscited by 6
  10. 2025HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modelingcited by 4
  11. 2025Can Past Experience Accelerate LLM Reasoning?cited by 3
  12. 2025Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Trainingcited by 3
  13. 2025PersonalAI: A Systematic Comparison of Knowledge Graph Storage and Retrieval Approaches for Personalized LLM agentscited by 2
  14. 2025DPRM: A Dual Implicit Process Reward Model in Multi-Hop Question Answeringcited by 2
  15. 2026MASRubric: Auditing Information Flow in Multi-Agent Systems with Failure-Distilled Pitfall Rubricscited by 2
  16. 2026MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QAcited by 2
  17. 2025Mixed-Precision Graph Neural Quantization for Low Bit Large Language Modelscited by 1
  18. 2025Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaborationcited by 1
  19. 2026Codified Finite-state Machines for Role-playingcited by 1
  20. 2026FLOWREADER: Min-Cost Flow Optimization for Multi-Modal Long Document Q&Acited by 1
  21. 2025FinReflectKG -- EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluationcited by 0
  22. 2025Graph-Guided Concept Selection for Efficient Retrieval-Augmented Generationcited by 0
  23. 2025InvertiTune: High-Quality Data Synthesis for Cost-Effective Single-Shot Text-to-Knowledge Graph Generationcited by 0
  24. 2026SOPRAG: Multi-view Graph Experts Retrieval for Industrial Standard Operating Procedurescited by 0
  25. 2026BOOKMARKS: Efficient Active Storyline Memory for Role-playingcited by 0
  26. 2026GRASP: Graph Agentic Search over Propositions for Multi-hop Question Answeringcited by 0
  27. 2026TRACE: State-Aware Query Processing over Temporal Evidence Graphs for Conversational Datacited by 0
  28. 2026Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Modelcited by 0
  29. 2026Learning to Retrieve Missing Evidence for Long-Term Memory QAcited by 0
  30. 2026From Knowledge Access to Source Learning: Developing Source-Specific Competencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Tasks
  2. 2026GABench: A Comprehensive Benchmark for Evaluating LLM Agents on Graph Analysis Tasks
  3. 2026GraphScout: Empowering Large Language Models with Intrinsic Exploration Ability for Agentic Graph Reasoning
  4. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis
  5. 2023Large Language Models on Graphs: A Comprehensive Survey
  6. 2026Are Large Language Models Suitable for Graph Computation? Progress and Prospects
  7. 2025Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning
  8. 2024GUNDAM: Aligning Large Language Models with Graph Understanding