# Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks

arXiv: https://arxiv.org/abs/2412.15605 · HTML: https://mlsanity.com/paper/2412.15605/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 11 of its 12 references that are in the map.
- First posted on arXiv 20 Dec 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 11 of 11)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [Efficiently Scaling Transformer Inference](https://mlsanity.com/paper/2211.05102/) (2022)
- [Long-context LLMs Struggle with Long In-context Learning](https://mlsanity.com/paper/2404.02060/) (2024)
- [Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach](https://mlsanity.com/paper/2407.16833/) (2024)
- [TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text](https://mlsanity.com/paper/2410.07590/) (2024)

## Built on by (most cited first, 30 of 38)

- [KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction](https://mlsanity.com/paper/2505.23416/) (2025)
- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answering](https://mlsanity.com/paper/2508.14052/) (2025)
- [How Many Instructions Can LLMs Follow at Once?](https://mlsanity.com/paper/2507.11538/) (2025)
- [Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs for Persistent Agents](https://mlsanity.com/paper/2603.04814/) (2026)
- [The Few-shot Dilemma: Over-prompting Large Language Models](https://mlsanity.com/paper/2509.13196/) (2025)
- [Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding](https://mlsanity.com/paper/2504.01281/) (2025)
- [Semantic Caching of Contextual Summaries for Efficient Question-Answering with Language Models](https://mlsanity.com/paper/2505.11271/) (2025)
- [Distilling Knowledge from Large Language Models: A Concept Bottleneck Model for Hate and Counter Speech Recognition](https://mlsanity.com/paper/2508.08274/) (2025)
- [Log-Augmented Generation: Scaling Test-Time Reasoning with Reusable Computation](https://mlsanity.com/paper/2505.14398/) (2025)
- [WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems](https://mlsanity.com/paper/2605.07068/) (2026)
- [KV Cache Translation across Heterogeneous Large Language Models](https://mlsanity.com/paper/2607.28979/) (2026)
- [Towards EnergyGPT: A Large Language Model Specialized for the Energy Sector](https://mlsanity.com/paper/2509.07177/) (2025)
- [MatKV: Trading Compute for Flash Storage in LLM Inference](https://mlsanity.com/paper/2512.22195/) (2025)
- [TableCache: Primary Foreign Key Guided KV Cache Precomputation for Low Latency Text-to-SQL](https://mlsanity.com/paper/2601.08743/) (2026)
- [Single-Round Vector RAG vs an LLM-Compiled Wiki: A Preregistered Comparison on a Small Multi-Domain Research Corpus](https://mlsanity.com/paper/2605.18490/) (2026)
- [LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding](https://mlsanity.com/paper/2606.04302/) (2026)
- [Leveraging Approximate Caching for Faster Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.05530/) (2025)
- [DangerMaps: Personalized Safety Advice for Travel in Urban Environments using a Retrieval-Augmented Language Model](https://mlsanity.com/paper/2503.14103/) (2025)
- [AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAM](https://mlsanity.com/paper/2510.17934/) (2025)
- [MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference](https://mlsanity.com/paper/2511.06010/) (2025)
- [Parallel Context-of-Experts Decoding for Retrieval Augmented Generation](https://mlsanity.com/paper/2601.08670/) (2026)
- [TEGRA: Text Encoding With Graph and Retrieval Augmentation for Misinformation Detection](https://mlsanity.com/paper/2602.11106/) (2026)
- [Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?](https://mlsanity.com/paper/2605.27494/) (2026)
- [Streaming Knowledge Compilation: Proactive Materiality-Scored Pinning for Time-Evolving LLM Wikis](https://mlsanity.com/paper/2606.09877/) (2026)
- [Parallel Key-Value Cache Fusion for Position Invariant RAG](https://mlsanity.com/paper/2501.07523/) (2025)
- [Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration](https://mlsanity.com/paper/2509.21530/) (2025)
- [Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding](https://mlsanity.com/paper/2509.21865/) (2025)
- [HA-RAG: Hotness-Aware RAG Acceleration via Mixed Precision and Data Placement](https://mlsanity.com/paper/2510.20878/) (2025)
- [Democratizing LLM Efficiency: From Hyperscale Optimizations to Universal Deployability](https://mlsanity.com/paper/2511.20662/) (2025)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) has gained traction as a powerful approach for enhancing language models by integrating external knowledge sources. However, RAG introduces challenges such as retrieval latency, potential errors in document selection, and increased system complexity. With the advent of large language models (LLMs) featuring significantly extended context windows, this paper proposes an alternative paradigm, cache-augmented generation (CAG) that bypasses real-time retrieval. Our method involves preloading all relevant resources, especially when the documents or knowledge for retrieval are of a limited and manageable size, into the LLM's extended context and caching its runtime parameters. During inference, the model utilizes these preloaded parameters to answer queries without additional retrieval steps. Comparative analyses reveal that CAG eliminates retrieval latency and minimizes retrieval errors while maintaining context relevance. Performance evaluations across multiple benchmarks highlight scenarios where long-context LLMs either outperform or complement traditional RAG pipelines. These findings suggest that, for certain applications, particularly those with a constrained knowledge base, CAG provide a streamlined and efficient alternative to RAG, achieving comparable or superior results with reduced complexity.