mlsanityData to 14 Oct 2026

Error Certificates for KV-Cache Eviction via Randomized Design

arXiv 2607.21475 ↗First posted 23 Jul 2026cs.LG cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Deterministic KV-cache eviction keeps the top-k tokens under an importance score and deletes the rest, and after the deletion the serving system cannot know what the eviction cost it on the current query. We replace the deterministic tail with Poisson sampling at known inclusion probabilities, which makes the eviction error identifiable and turns a survey-sampling variance estimator over the retained set into a per-step error certificate at one extra scalar per retained token. On a thirty-turn assistant compressed to a 10% cache budget, the certificate-gated system answers 0.97 of recall questions against 0.09 for top-k, and for facts stated 26 to 30 turns earlier it recalls 97% against 2%. We prove that no estimator computable from the information a deterministic scheme retains is consistent for its own eviction error: evicted values can be altered so that everything retained is unchanged while the true attention-output error grows without bound. Under the Poisson design the certificate covers the realized attention error in 96.9--97.7% of 12,096 replay cells and in 98.1--99.7% on twelve further architectures. Randomization buys attribution, not prediction: a pre-registered study on LongBench at 6k and 16k tokens (about 74,000 generations) finds question-aware eviction at 25--50% budgets nearly free and output log-probability the better failure predictor, while the certificate answers the question confidence cannot, separating eviction-induced from inherent failures at AUC 0.65--0.75 against 0.47--0.54, and schedules recomputation at 1.7--1.8 times the gain of random gating. On real long-term conversations the gated system returns the full-cache score inside the heavy-damage regime, and the rule that triggers it is the same across five model families.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

25 of 39 references.

References · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2023Mistral 7Bcited by 3,164
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  9. 2017Selective Classification for Deep Neural Networkscited by 852
  10. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  11. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  12. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
13 more
  1. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  2. 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 229
  3. 2020Estimating means of bounded random variables by bettingcited by 169
  4. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 155
  5. 2024Transformers are Multi-State RNNscited by 120
  6. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  7. 2025CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Evictioncited by 13
  8. 2026Value-Aware Stochastic KV Cache Eviction for Reasoning Modelscited by 8
  9. 2025vAttention: Verified Sparse Attentioncited by 6
  10. 2026MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inferencecited by 6
  11. 2026When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compressioncited by 5
  12. 2014Generalized friendship paradox in complex networks: The case of scientific collaborationcited by 2
  13. 2026Forget Without Compromise: Nexus Sampling for Streaming KV-Cache Eviction Under Fixed Budgetscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Probabilistic Interpretation of KV Cache Eviction
  2. 2026AvoKV-E: Payload-Aware KV Cache Eviction for Long Reasoning
  3. 2026Trust the Mass: Forced Weights in KV-Cache Eviction
  4. 2026PAGE: Partition-Aware Gated KV-Cache Eviction
  5. 2026Randomization Boosts KV Caching, Learning Balances Query Load: A Joint Perspective
  6. 2026Monte Carlo Estimation for KV Cache Eviction
  7. 2026Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
  8. 2026KV2: A Self-Refining KV Cache