mlsanityData to 14 Oct 2026

SAFE-KV: Causal Suffix Repair for Behavior-Preserving Non-Prefix KV Cache Reuse in RAG

arXiv 2610.15186 ↗First posted 10 Oct 2026cs.CL cs.LGcited by 0

On the map: Retrieval-augmented generation → Generation and context integration also fits: RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-augmented generation (RAG) reuses the same documents across many requests, making non-prefix KV-cache reuse attractive. Yet a cached document state is not reusable text: it is a context-conditioned causal state whose keys and values depend on the prefix, position, and preceding documents under which it was computed. We formulate non-prefix KV reuse as a behavior-preservation problem with full prefill on the exact prompt as the reference, and introduce SAFE-KV, which repairs a composed cache by re-encoding the causal suffix from a selected start document. On Qwen2.5-7B-Instruct, prefix-conditioned composition preserves only 47.2% of HotpotQA-500 and 37.8% of 2Wiki-500 full-prefill answers; SAFE-KV with risk-guided K2 selection restores 99.8%/98.4% (donor-copy protocol) and 100.0%/98.4% with actual suffix forwarding. Document-only and fixed-window repairs recover far less, because the document states after the repaired region keep histories that omit it. Reuse without repair is exact up to four documents and degrades at sixteen, where suffix repair recovers 98-100%. Faithful repair re-encodes most of the context, so in a single-request L40S study SAFE-KV's operating rule serves every workload by full prefill. SAFE-KV thus establishes the causal-state contract for non-prefix reuse and a validated repair mechanism that meets it.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 66 of its 96 references that are in the map. First posted on arXiv 10 Oct 2026.

Builds on

66 of 96 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text10 Oct
  2. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context2 Oct
  3. 2023REPLUG: Retrieval-Augmented Black-Box Language Models30 Jan
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Models5 Aug
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  10. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  11. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
38 more
  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  12. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  13. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  14. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  15. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  16. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  17. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  18. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  19. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  20. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  21. 2022Mass-Editing Memory in a Transformercited by 1,001
  22. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  23. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  24. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  25. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  26. 2019Passage Re-ranking with BERTcited by 645
  27. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  28. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  29. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  30. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  31. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  32. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  33. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  34. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  35. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  36. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  37. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  38. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving
  2. 2026KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse
  3. 2026Budgeted KV-Cache Repair for Evolving Contexts
  4. 2026PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts
  5. 2026Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents
  6. 2026Fine-Tuning a KV Cache Concatenation-Aware Model or Recomputing KV Caches? Why Not Both?
  7. 2026RelaxKV: Recomputation Guided by the Query with Sparse Context Attention for Efficient KV Cache Reuse
  8. 2026Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?