mlsanityData to 14 Oct 2026

Accelerating Adaptive Retrieval Augmented Generation via Instruction-Driven Representation Reduction of Retrieval Overlaps

arXiv 2505.12731 ↗First posted 19 May 2025cs.AIcited by 6

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-augmented generation (RAG) has emerged as a pivotal method for expanding the knowledge of large language models. To handle complex queries more effectively, researchers developed Adaptive-RAG (A-RAG) to enhance the generated quality through multiple interactions with external knowledge bases. Despite its effectiveness, A-RAG exacerbates the pre-existing efficiency challenges inherent in RAG, which are attributable to its reliance on multiple iterations of generation. Existing A-RAG approaches process all retrieved contents from scratch. However, they ignore the situation where there is a significant overlap in the content of the retrieval results across rounds. The overlapping content is redundantly represented, which leads to a large proportion of repeated computations, thus affecting the overall efficiency. To address this issue, this paper introduces a model-agnostic approach that can be generally applied to A-RAG methods, which is dedicated to reducing the redundant representation process caused by the overlapping of retrieval results. Specifically, we use cache access and parallel generation to speed up the prefilling and decoding stages respectively. Additionally, we also propose an instruction-driven module to further guide the model to more effectively attend to each part of the content in a more suitable way for LLMs. Experiments show that our approach achieves 2.79 and 2.33 times significant acceleration on average for prefilling and decoding respectively while maintaining equal generation quality.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 39 of its 51 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

39 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting11 Jul

Other references · 38

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  11. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  12. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
26 more
  1. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  2. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  4. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  5. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  6. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  7. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  8. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  9. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  10. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 234
  11. 2021Internet-Augmented Dialogue Generationcited by 217
  12. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  13. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  14. 2023Speculative Decoding with Big Little Decodercited by 164
  15. 2023Accelerating LLM Inference with Staged Speculative Decodingcited by 151
  16. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  17. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 122
  18. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  19. 2020Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Searchcited by 76
  20. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Textcited by 63
  21. 2024When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentationcited by 59
  22. 2020IIRC: A Dataset of Incomplete Information Reading Comprehension Questionscited by 53
  23. 2022Token Dropping for Efficient BERT Pretrainingcited by 42
  24. 2023Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decodingcited by 40
  25. 2023Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learningcited by 29
  26. 2024Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoEcited by 2
  2. 2025Mesh RAG: Retrieval Augmentation for Autoregressive Mesh Generationcited by 1
  3. 2026From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priorscited by 1
  4. 2025HA-RAG: Hotness-Aware RAG Acceleration via Mixed Precision and Data Placementcited by 0
  5. 2026Small Language Model Helps Resolve Semantic Ambiguity of LLM Promptcited by 0
  6. 2026Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunkingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control
  2. 2024Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks
  3. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices
  4. 2024Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systems
  5. 2024Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data
  6. 2024Toward Optimal Search and Retrieval for RAG
  7. 2024XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
  8. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions