mlsanityData to 7 Oct 2026

AvoKV-E: Payload-Aware KV Cache Eviction for Long Reasoning

arXiv 2610.03007 ↗First posted 2 Oct 2026cs.LG cs.AIcited by 0

On the map: Efficient LLM inference and serving → Efficient reasoning and agent inference also fits: KV cache optimization

Abstract (arXiv)

Long-output reasoning shifts the KV-cache bottleneck from the fixed prompt to the generated trace. Existing reasoning-cache eviction methods largely treat cached entries as routing objects, estimating whether an old key will still be read, will recur, or can be replaced. This routing-only view overlooks two effects: low-attention entries can carry large value payloads whose removal changes future predictions, and newly generated states can appear stale before later queries have had a chance to read them. We introduce AvoKV-E, a training-free eviction policy that first delays eligibility for recent states and then ranks eligible entries using candidate-normalized read pressure, key redundancy, and value-payload potential. According to empirical evaluation across different models and datasets, AvoKV-E matches or exceeds redundancy-aware, recurrence-based, and thought-adaptive eviction baselines at matched active-KV budgets, with its largest gains in the tightest-cache regime. Component and counterfactual analyses further connect these gains to delayed observation, payload-aware scoring, redundancy, and scale-robust normalization. Together, the results show that long-reasoning KV eviction should preserve not only keys that are likely to be read, but also the value payloads that sustain the reasoning trajectory.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient reasoning and agent inference, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 23 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

18 of 23 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026LongFlow: Efficient KV Cache Compression for Reasoning Models12 Mar
  2. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning19 Jun
  3. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Models30 May

Other references · 15

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  6. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  7. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  8. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 466
  9. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  10. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 283
  11. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  12. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
3 more
  1. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoningcited by 96
  2. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Timecited by 43
  3. 2025PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMscited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Value-Aware Stochastic KV Cache Eviction for Reasoning Models
  2. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
  3. 2026Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
  4. 2026A Probabilistic Interpretation of KV Cache Eviction
  5. 2026ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution
  6. 2025SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models
  7. 2026Monte Carlo Estimation for KV Cache Eviction
  8. 2026Back from the Future: Key-Value Cache Management by Counter-Causal Surprise