mlsanityData to 7 Oct 2026

RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation

arXiv 2310.04408 ↗First posted 6 Oct 2023cs.CLcited by 253

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Retrieving documents and prepending them in-context at inference time improves performance of language model (LMs) on a wide range of tasks. However, these documents, often spanning hundreds of words, make inference substantially more expensive. We propose compressing the retrieved documents into textual summaries prior to in-context integration. This not only reduces the computational costs but also relieves the burden of LMs to identify relevant information in long retrieved documents. We present two compressors -- an extractive compressor which selects useful sentences from retrieved documents and an abstractive compressor which generates summaries by synthesizing information from multiple documents. Both compressors are trained to improve LMs' performance on end tasks when the generated summaries are prepended to the LMs' input, while keeping the summary concise.If the retrieved documents are irrelevant to the input or offer no additional information to LM, our compressor can return an empty string, implementing selective augmentation.We evaluate our approach on language modeling task and open domain question answering task. We achieve a compression rate of as low as 6% with minimal loss in performance for both tasks, significantly outperforming the off-the-shelf summarization models. We show that our compressors trained for one LM can transfer to other LMs on the language modeling task and provide summaries largely faithful to the retrieved documents.

Key facts. Cited by 253 papers in the map (data through 7 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 44 of its 64 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

44 of 64 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context2 Oct
  2. 2023In-Context Retrieval-Augmented Language Models31 Jan
  3. 2023REPLUG: Retrieval-Augmented Black-Box Language Models30 Jan
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Models5 Aug
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May

Other references · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2016Pointer Sentinel Mixture Modelscited by 2,992
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
27 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  2. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  3. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  4. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  5. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  6. 2015Teaching Machines to Read and Comprehendcited by 1,039
  7. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  8. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  9. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  10. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  11. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  12. 2023Adapting Language Models to Compress Contextscited by 350
  13. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  14. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  15. 2022Prompting GPT-3 To Be Reliablecited by 258
  16. 2022Learning by Distilling Contextcited by 128
  17. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
  18. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Modelscited by 96
  19. 2021Efficient Nearest Neighbor Language Modelscited by 93
  20. 2023Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)cited by 40
  21. 2023Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learningcited by 40
  22. 2022Prompt Injection: Parameterization of Fixed Inputscited by 35
  23. 2022Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarizationcited by 32
  24. 2023Propagating Knowledge Updates to LMs Through Distillationcited by 28
  25. 2022Chunk-based Nearest Neighbor Machine Translationcited by 21
  26. 2021Decision-Focused Summarizationcited by 11
  27. 2023Concise Answers to Complex Questions: Summarization of Long-form Answerscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 89; 2025: 83; 2026: 75.

253 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  5. 2024Tool Learning with Large Language Models: A Surveycited by 221
  6. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  7. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  8. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  9. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
  10. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Tokencited by 123
  11. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 101
  12. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
38 more
  1. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  2. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  3. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  4. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  5. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  6. 2023In-context Pretraining: Language Modeling Beyond Document Boundariescited by 76
  7. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  8. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  9. 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
  10. 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 68
  11. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  12. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
  13. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 51
  14. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  15. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedbackcited by 46
  16. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  17. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  18. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 46
  19. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  20. 2024Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Modelscited by 41
  21. 2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitionscited by 38
  22. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  23. 2024Online Adaptation of Language Models with a Memory of Amortized Contextscited by 35
  24. 2024Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMscited by 34
  25. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  26. 2024Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refusecited by 33
  27. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  28. 2025Removal of Hallucination on Hallucination: Debate-Augmented RAGcited by 29
  29. 2026SkillReducer: Optimizing LLM Agent Skills for Token Efficiencycited by 29
  30. 2024BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidencecited by 28
  31. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answeringcited by 28
  32. 2024PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agentscited by 28
  33. 2024Retrieval-augmented generation in multilingual settingscited by 27
  34. 2025From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agentscited by 27
  35. 2024Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguitycited by 26
  36. 2024Context Embeddings for Efficient Answer Generation in RAGcited by 26
  37. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  38. 2024Open-RAG: Enhanced Retrieval-Augmented Reasoning with Open-Source Large Language Modelscited by 25

50 most cited of 253 All 253 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ACoRN: Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models
  2. 2025Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models
  3. 2024EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation
  4. 2026EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering
  5. 2024CompAct: Compressing Retrieved Documents Actively for Question Answering
  6. 2024Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
  7. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference
  8. 2024Enhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor