mlsanityData to 14 Oct 2026

Budgeted KV-Cache Repair for Evolving Contexts

arXiv 2610.17338 ↗First posted 12 Oct 2026cs.AIcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization also fits: Efficient reasoning and agent inference

Abstract (arXiv)

Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computation. Later key-value (KV) states remain conditioned on the deleted content, while recomputing the entire retained suffix can be costly. We introduce budgeted KV-cache repair: recovering task performance by recomputing a user-chosen fraction of the retained suffix. Comparing repair allocations under identical relevance scores and token budgets, we find that refreshing coherent regions can substantially outperform refreshing isolated tokens. This finding motivates Semantic-Unit Refresh (SUR), which allocates the budget to query-relevant text regions using inexpensive embedding scores, without training or an auxiliary transformer forward. To execute these repairs efficiently, a layer-packed executor processes selected states together at each layer while preserving full causal cache access. Across five models, SUR achieves strong recovery in controlled deletion tasks while recomputing only 2-5% of the suffix. On four natural multi-hop QA datasets, it achieves near-full-recompute exact match at a 5% budget with lower post-edit latency. The executor accelerates fragmented refresh by up to 16.1x over interval-wise execution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 28 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

22 of 28 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing15 Jun
  2. 2026Adaptive KV Cache Reuse for Fast Long-Context LLM Serving20 May
  3. 2025A3: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving13 Nov
  4. 2024Let the Code LLM Edit Itself When You Edit the Code3 Jul
  5. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion26 May
  6. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr

Other references · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  8. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  9. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  10. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  11. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  12. 2020IIRC: A Dataset of Incomplete Information Reading Comprehension Questionscited by 53
4 more
  1. 2026MEMENTO: Teaching LLMs to Manage Their Own Contextcited by 18
  2. 2026Models Take Notes at Prefill: KV Cache Can Be Editable and Composablecited by 6
  3. 2026InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Contextcited by 3
  4. 2026QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Servingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits
  2. 2026PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts
  3. 2026RelaxKV: Recomputation Guided by the Query with Sparse Context Attention for Efficient KV Cache Reuse
  4. 2026RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
  5. 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation
  6. 2025KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction
  7. 2026AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems
  8. 2026CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters