mlsanityData to 7 Oct 2026

RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval

arXiv 2401.18059 ↗First posted 31 Jan 2024cs.CL cs.LGcited by 519

On the map: Retrieval-augmented generation → Knowledge-graph and structured-knowledge RAG

Abstract (arXiv)

Retrieval-augmented language models can better adapt to changes in world state and incorporate long-tail knowledge. However, most existing methods retrieve only short contiguous chunks from a retrieval corpus, limiting holistic understanding of the overall document context. We introduce the novel approach of recursively embedding, clustering, and summarizing chunks of text, constructing a tree with differing levels of summarization from the bottom up. At inference time, our RAPTOR model retrieves from this tree, integrating information across lengthy documents at different levels of abstraction. Controlled experiments show that retrieval with recursive summaries offers significant improvements over traditional retrieval-augmented LMs on several tasks. On question-answering tasks that involve complex, multi-step reasoning, we show state-of-the-art results; for example, by coupling RAPTOR retrieval with the use of GPT-4, we can improve the best performance on the QuALITY benchmark by 20% in absolute accuracy.

Key facts. Cited by 519 papers in the map (data through 7 Oct 2026). Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation. Builds on 50 of its 61 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

50 of 61 references.

References · 50

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
38 more
  1. 2017Billion-scale similarity search with GPUscited by 2,477
  2. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  3. 2019Language Models as Knowledge Bases?cited by 1,922
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  5. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  6. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  8. 2019How Can We Know What Language Models Know?cited by 953
  9. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  10. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  11. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  12. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  13. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  14. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  15. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  16. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  17. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  18. 2022Memory-Based Model Editing at Scalecited by 443
  19. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  20. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  21. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
  22. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  23. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  24. 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
  25. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  26. 2023CoLT5: Faster Long-Range Transformers with Conditional Computationcited by 84
  27. 2018Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervisedcited by 75
  28. 2022Recitation-Augmented Language Modelscited by 65
  29. 2023Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Studycited by 63
  30. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  31. 2023A Survey on Long Text Modeling with Transformerscited by 47
  32. 2021Joint Passage Ranking for Diverse Multi-Answer Retrievalcited by 43
  33. 2022Nonparametric Masked Language Modelingcited by 39
  34. 2021Dense Hierarchical Retrieval for Open-Domain Question Answeringcited by 33
  35. 2022Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarizationcited by 32
  36. 2022HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarizationcited by 22
  37. 2020Frustratingly Hard Evidence Retrieval for QA Over Bookscited by 7
  38. 2017Contextualizing Citations for Scientific Summarization using Word Embeddings and Domain Knowledgecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 62; 2025: 148; 2026: 309.

519 papers.

Sort:

By area:

  1. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  2. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  3. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  4. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  5. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  6. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  7. 2025From RAG to Memory: Non-Parametric Continual Learning for Large Language Modelscited by 223
  8. 2025LightMem: Lightweight and Efficient Memory-Augmented Generationcited by 163
  9. 2024Language agents achieve superhuman synthesis of scientific knowledgecited by 121
  10. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  11. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  12. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
38 more
  1. 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
  2. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  3. 2024From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMscited by 77
  4. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  5. 2024KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generationcited by 67
  6. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  7. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  8. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 51
  9. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarioscited by 49
  10. 2025LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corporacited by 47
  11. 2025Retrieval-Augmented Generation with Hierarchical Knowledgecited by 46
  12. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 46
  13. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 45
  14. 2024Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversationscited by 43
  15. 2024GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Modelscited by 42
  16. 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 42
  17. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  18. 2024Online Adaptation of Language Models with a Memory of Amortized Contextscited by 35
  19. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  20. 2025ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generationcited by 27
  21. 2025From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agentscited by 27
  22. 2026TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agentscited by 26
  23. 2024TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervisioncited by 25
  24. 2025NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodescited by 25
  25. 2025HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generationcited by 24
  26. 2025GraphRAG-Bench: Challenging Domain-Specific Reasoning for Evaluating Graph Retrieval-Augmented Generationcited by 24
  27. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfacescited by 24
  28. 2024SiReRAG: Indexing Similar and Related Information for Multihop Reasoningcited by 23
  29. 2024OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Modelscited by 23
  30. 2025AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corporacited by 23
  31. 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
  32. 2025CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehensioncited by 23
  33. 2025Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answeringcited by 22
  34. 2025PropRAG: Guiding Retrieval with Beam Search over Proposition Pathscited by 22
  35. 2024Self-Updatable Large Language Models by Integrating Context into Model Parameterscited by 21
  36. 2025SkillFlow: Scalable and Efficient Agent Skill Retrieval Systemcited by 20
  37. 2025Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challengescited by 19
  38. 2026Adaptive Memory Admission Control for LLM Agentscited by 19

50 most cited of 519 All 519 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Tree Navigation Without LLM Summaries: A Matched-Cost Study of Hierarchical Retrieval for Long-Document QA
  2. 2026Latent Abstraction for Retrieval-Augmented Generation
  3. 2024Hierarchical Retrieval-Augmented Generation Model with Rethink for Multi-hop Question Answering
  4. 2025Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval
  5. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
  6. 2024Tree of Reviews: A Tree-based Dynamic Iterative Retrieval Framework for Multi-hop Question Answering
  7. 2024Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs
  8. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approach