mlsanityData to 14 Oct 2026

Enhancing RAG Efficiency with Adaptive Context Compression

arXiv 2507.22931 ↗First posted 24 Jul 2025cs.CL cs.AIcited by 8

On the map: Retrieval-augmented generation → Generation and context integration

also in 1 more
Abstract (arXiv)

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but incurs significant inference costs due to lengthy retrieved contexts. While context compression mitigates this issue, existing methods apply fixed compression rates, over-compressing simple queries or under-compressing complex ones. We propose Adaptive Context Compression for RAG (ACC-RAG), a framework that dynamically adjusts compression rates based on input complexity, optimizing inference efficiency without sacrificing accuracy. ACC-RAG combines a hierarchical compressor (for multi-granular embeddings) with a context selector to retain minimal sufficient information, akin to human skimming. Evaluated on Wikipedia and five QA datasets, ACC-RAG outperforms fixed-rate methods and matches/unlocks over 4 times faster inference versus standard RAG while maintaining or improving accuracy.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 42 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

28 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Context Embeddings for Efficient Answer Generation in RAG12 Jul
  2. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token22 May
  3. 2023REPLUG: Retrieval-Augmented Black-Box Language Models30 Jan
  4. 2022Neural Machine Translation with Contrastive Translation Memories6 Dec
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 22

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  9. 2023Large Language Modelscited by 780
  10. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  11. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  12. 2023Learning to Compress Prompts with Gist Tokenscited by 390
10 more
  1. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  2. 2023Adapting Language Models to Compress Contextscited by 356
  3. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  4. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 297
  5. 2020Nearest Neighbor Machine Translationcited by 209
  6. 2023Augmenting Language Models with Long-Term Memorycited by 189
  7. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  8. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Modelscited by 97
  9. 2024Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answeringcited by 96
  10. 2024Prompt Compression for Large Language Models: A Surveycited by 67

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inferencecited by 9
  2. 2026PoC: Performance-oriented Context Compression for Large Language Models via Performance Predictioncited by 3
  3. 2026ZipRL: Adaptive Multi-Turn Context Compression with Hindsight Response Replaycited by 3
  4. 2026Who Benefits from RAG? The Role of Exposure, Utility and Attribution Biascited by 1
  5. 2026Large Language Models Do Not Always Need Readable Languagecited by 1
  6. 2026A Unified Model and Document Representation for On-Device Retrieval-Augmented Generationcited by 0
  7. 2026Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAGcited by 0
  8. 2026REVA: Reusable Evidence View Aggregation for Context-Efficient RAG Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Less Is More: Elevating RAG via Performance-Driven Context Compression
  2. 2024Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
  3. 2025SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression
  4. 2026Rethinking Soft Compression in Retrieval-Augmented Generation: A Query-Conditioned Selector Perspective
  5. 2024Lighter And Better: Towards Flexible Context Adaptation For Retrieval Augmented Generation
  6. 2025Accelerating Adaptive Retrieval Augmented Generation via Instruction-Driven Representation Reduction of Retrieval Overlaps
  7. 2026L-RAG: Balancing Context and Retrieval with Entropy-Based Lazy Loading
  8. 2024AdaComp: Extractive Context Compression with Adaptive Predictor for Retrieval-Augmented Large Language Models