mlsanityData to 7 Oct 2026

WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Minds

arXiv 2610.02713 ↗First posted 2 Oct 2026cs.CL cs.AR cs.DCcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Most KV-cache compression methods classify attention heads once, either offline or during prefill, and keep this classification fixed throughout generation. Across three models (1.5B-8B) and three regimes (needle retrieval, long chain-of-thought, and multi-turn recall), we measure head behavior on four model-regime combinations and find that most heads change their reading behavior at least once during generation. We introduce WakeKV, a reactive residency policy that moves cooling heads to a recoverable CPU reservoir rather than freezing or permanently evicting their state. At matched memory or budget, WakeKV consistently improves miss rate over frozen classification and destructive eviction, evaluated across five model-regime combinations and over three cited baselines (SnapKV, uniform R-KV, and ReasonAlloc) across four eligible combinations. A FlexiCache/vLLM implementation on Mistral-7B confirms the benefit on real hardware, improving throughput while retaining LongBench quality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 22 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

21 of 22 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference20 Jan
  2. 2025G-KV: Decoding-Time KV Cache Eviction with Global Attention29 Nov
  3. 2025FlexiCache: Leveraging Temporal Stability of Attention Heads for Efficient KV Cache Management2 Nov
  4. 2025LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation11 Sep
  5. 2025CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences16 Mar
  6. 2025HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading18 Feb
  7. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs19 Dec
  8. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation18 Dec
  9. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference28 Oct
  10. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct
  11. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
  12. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Heads22 Jul
3 more
  1. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  2. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management28 Jun
  3. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr

Other references · 6

  1. 2023Let's Verify Step by Stepcited by 3,598
  2. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 52
  3. 2024Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengthscited by 44
  4. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Modelscited by 21
  5. 2026ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Modelscited by 3
  6. 2026Retrieval Heads are Dynamiccited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression
  2. 2026Minima-KV: Retention-Preserving KV Cache Compression with Mixed-Format Paged Attention
  3. 2025SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
  4. 2026Beyond Sparse Weights: When Is Attention Compressible?
  5. 2026Small Frequency Corrections Can Change What Survives KV Cache Compression
  6. 2026HeadQ: Model-Visible Distortion and Score-Space Correction for KV-Cache Quantization
  7. 2026LoopGuard: Breaking Self-Reinforcing Attention Loops via Dynamic KV Cache Intervention
  8. 2026From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs