mlsanityData to 14 Oct 2026

Budgeted Cache Repair for Cross-Context KV-Cache Reuse

arXiv 2610.02233 ↗First posted 27 Sep 2026cs.AR cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Cross-context KV-cache reuse predicts a shared segment's keys and values under a new prefix instead of recomputing them, and has been reported to do so without quality loss. We find otherwise, and identify two problems. (1) A hidden cost: on MMLU and GSM8K, reuse costs substantial accuracy. (2) A decision at the wrong unit: no rule for deciding whether to reuse a cache removes that cost. What does help is choosing which parts of the cache to recompute, and the value of choosing well falls as the unit of choice grows: informed selection removes 49.5% of the cache error beyond chance at single rows (one token's keys and values), 10.6% at 64-token chunks, and nothing at the level of whole calls. Budgeted Cache Repair (BCR) acts at the unit where selection still pays. It drafts two tokens from the assembled cache, ranks cache rows by the attention those tokens pay them, and recomputes a fixed budget of rows exactly, in one of three layouts. The cost is paid rather than predicted away, and the draft that fails as a gate succeeds as a selector. BCR restores GSM8K to dense-prefill accuracy while still serving most calls from cache, and its best layout outperforms every reuse baseline's mean in the reference grid. The draft also beats a coin-flip selector at the same budget - a control prior evaluations lack.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 25 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

24 of 25 references.

References · 24

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  9. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  10. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  11. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  12. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
12 more
  1. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  2. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  3. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  4. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 141
  5. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
  6. 2024Language Agents as Optimizable Graphscited by 89
  7. 2025KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systemscited by 40
  8. 2026Fast KV Compaction via Attention Matchingcited by 31
  9. 2025CacheClip: Accelerating RAG with Effective KV Cache Reusecited by 16
  10. 2025Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Querycited by 14
  11. 2026ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generationcited by 13
  12. 2026InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Contextcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs
  2. 2026Evaluating the accuracy of KV cache reuse techniques
  3. 2026RelaxKV: Recomputation Guided by the Query with Sparse Context Attention for Efficient KV Cache Reuse
  4. 2025KV Cache Recycling to Expand Usable Context Capacity in Low Parameter LLMs
  5. 2026Budgeted KV-Cache Repair for Evolving Contexts
  6. 2026To Cache or Not to Cache: Understanding and Mitigating Safety Incompatibility in Cross-Model KV Cache Reuse
  7. 2026KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse
  8. 2026A Universal Context-Reuse Layer for Cross-Model KV Sharing