mlsanityData to 7 Oct 2026

PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts

arXiv 2609.26219 ↗First posted 18 Aug 2026cs.DC cs.LGcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Long-running LLM agent workflows often revise interior context spans while retaining long suffixes. Although suffix tokens remain unchanged, altered causal histories and rotary positions prevent exact reuse of their offloaded key-value (KV) states. Full suffix recomputation wastes prefill work, while indiscriminate reuse propagates stale states and full-precision restoration adds data movement. We present PatchKV, a profile-guided recovery system for suffix-preserving revisions. PatchKV decomposes adjacent context versions into an exact prefix, an updated span, and an aligned suffix. It predicts an edit-local dirty region using an offline length-conditioned drift model, augments this region with sparse nonlocal blocks selected from stored attention, and block-rounds their union into a fixed repair set. The remaining suffix blocks are restored from CPU memory using frozen per-block precision tags and a fused path for dequantization, RoPE correction, and KV-page placement. Across three models and three long-context question-answering workloads, PatchKV achieves a 2.51-3.85× speedup in mean resume time-to-first-token over full suffix recomputation and a 1.26-2.06× speedup over CacheBlend, while matching or exceeding CacheBlend's F1 score in six of nine settings and remaining within 1.36 points in the others.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 35 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

30 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug

Other references · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  12. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
17 more
  1. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  3. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  4. 2019Attention is not Explanationcited by 916
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  6. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  7. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  8. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  9. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  10. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  11. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  12. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  13. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  14. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  15. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  16. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  17. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits
  2. 2026KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
  3. 2026AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems
  4. 2026PatchKV: Weight-Space Compensation of KV Cache
  5. 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation
  6. 2026RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
  7. 2026KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
  8. 2026KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints