mlsanityData to 14 Oct 2026

Small Frequency Corrections Can Change What Survives KV Cache Compression

arXiv 2608.27128 ↗First posted 27 Aug 2026cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Compressing a key-value cache before its next question is known requires choosing what to retain without knowing which evidence will matter. Value energy measures entry strength but does not distinguish isolated keys from those with many similar neighbors. We introduce TwinKV, a training-free method that discounts value energy by nonlocal post-RoPE key frequency. Prefix attention allocates head capacities, while retained entries preserve their original keys and values under an exact storage budget. Across four language models, TwinKV exceeds five evaluated compressed baselines in mean score on LongBench, LooGLE, and RULER at 50% KV removal. Component controls isolate the frequency contribution. On Llama-3.2-1B RULER at 75% removal, normalized frequency weights average 0.95, yet change 7% of nonprotected retained positions and improve value-only retention by about 5.5 points under both uniform and adapted capacities. Permuting the weights within heads weakens this gain. These results show that modest frequency corrections can change retention and answering outcomes, with effects that depend on the model and task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 22 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

21 of 22 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026NestedKV: Nested Memory Routing for Long-Context KV Cache Compression26 May
  2. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution1 Oct
  3. 2025Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs4 Jun
  4. 2025FlowKV: Enhancing Multi-Turn Conversational Coherence in LLMs via Isolated Key-Value Cache Management21 May
  5. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference1 Feb
  6. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  7. 2024Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters18 Jun
  8. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  10. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  11. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 10

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  6. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  7. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 202
  8. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 54
  9. 2025Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compressioncited by 16
  10. 2026SONIC: Segmented Optimized Nexus for Information Compression in Key-Value Cachingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compression
  2. 2025Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques
  3. 2026Back from the Future: Key-Value Cache Management by Counter-Causal Surprise
  4. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
  5. 2025REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression
  6. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression
  7. 2026Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
  8. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation