mlsanityData to 14 Oct 2026

Minimal-Intervention KV Retention via Set-Conditioned Diversity

arXiv 2605.14292 ↗First posted 14 May 2026cs.LG cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

KV-cache compression at small budgets is a crowded design space spanning cache representation, head-wise routing, compression cadence, decoding behavior, and within-budget scoring. We study seven mechanisms across these five families under matched mean cache on long-form mathematical reasoning (MATH-500~citehendrycks2021math) with two distilled-reasoning models (Qwen-7B and Llama-8B variants of DeepSeek-R1-Distill~citedeepseek2025r1) at budgets b ∈ 64, 128. All seven were rejected. We then propose α, a one-function modification to the TriAttention~citemao2026triattention retention scorer that replaces argmax-top-k with greedy facility-location-inspired selection under a V-space redundancy penalty controlled by a single weight λ. A pre-registered protocol tunes λ on a frozen development split and confirms on a disjoint held-out split; with λ = 0.5, α clears Bonferroni on two of the four (model, budget) cells (Qwen b=128 and Llama b=64), no cell is significantly negative, and the pre-registered Branch~A triggers. The finding is asymmetric: a minimal scoring modification beat heavier structural redesigns in this regime, and the combined matched-memory, sympy-graded, held-out confirmation protocol is the evidence standard that made the asymmetry visible.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 20 of its 27 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

20 of 27 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache27 Nov
  2. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  3. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  4. 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attention21 May
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  6. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  7. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 13

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  6. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  7. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  9. 2021Accounting for Variance in Machine Learning Benchmarkscited by 201
  10. 2024QAQ: Quality Adaptive Quantization for LLM KV Cachecited by 66
  11. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 54
  12. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 40
1 more
  1. 2026TriAttention: Efficient Long Reasoning with Trigonometric KV Compressioncited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Behavior-Preserving KV Cache Compression
  2. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
  3. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
  4. 2026CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache
  5. 2026CASK: Core-Aware Selective KV Compression for Reasoning Traces
  6. 2026iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD
  7. 2026ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression
  8. 2026Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning