Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits
On the map: Retrieval-augmented generation → Generation and context integration
also in 2 more
Abstract (arXiv)
KV-cache reuse can reduce inference cost in retrieval-augmented generation and agentic systems, but cached contexts may become stale when retrieved knowledge, working memory, or user state is edited. Under causal self-attention, even a local edit can affect downstream KV states. A full re-prefill reliably restores consistency but is costly, whereas refreshing only the edited span can leave downstream dependencies stale. We formulate in-place repair as budgeted recomputation and compare training-free position-selection policies on a factual RAG benchmark with matched direct and derived edits. Across three model families, all policies repair direct cases, but derived cases clearly separate them. At the primary budget, a contiguous edit-local window recovers at least 0.94 of the post-edit answer margin and substantially outperforms attention-based, KV-deviation, and structural selectors. Mechanistic analysis shows that position sets effective under clean-state transplantation can fail under actual recomputation because scattered positions inherit surrounding staleness. The edit-local advantage also depends on adjacency and largely disappears when the answer-bearing text moves downstream. Because answer-relevant edits almost always corrupt model behavior, failure severity is difficult to predict, and repair is 13-21 times faster than full re-prefill, our results support unconditional edit-local repair when the dependent text remains adjacent to the edit.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 8 of its 14 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
8 of 14 references.
Closest ancestors · 1
Same direction, newest first
- 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text10 Oct
Other references · 7
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
- 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
- 2025KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reusecited by 22
- 2026KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasingcited by 6
- 2026Leyline: KV Cache Directives for Agentic Inferencecited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Budgeted KV-Cache Repair for Evolving Contexts
- 2026PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts
- 2026Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents
- 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation
- 2026SemPIC: Learning Semantic Position-Independent KV Caches
- 2026CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion
- 2026AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems
- 2026RelaxKV: Recomputation Guided by the Query with Sparse Context Attention for Efficient KV Cache Reuse