mlsanityData to 7 Oct 2026

More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression

arXiv 2602.02199 ↗First posted 2 Feb 2026cs.AI cs.CLcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

While Large Language Models (LLMs) can theoretically support extensive context windows, their actual deployment is constrained by the linear growth of Key-Value (KV) cache memory. Prevailing compression strategies mitigate this through various pruning mechanisms, yet trade-off semantic recall for memory efficiency. In this work, we present LASER-KV (Layer Accumulated Selection with Exact-LSH Recall), a framework designed to test the limits of KV compression under a strict accumulative budgeting policy. We deviate from the standard fixed summary size approach by implementing a block-wise accumulation strategy governed by a protection divisor (n). This allows us to isolate the effects of compression from sliding window artifacts. Our experiments on the Babilong benchmark reveal performance degradation in previous compression methods by 15-30% on various long context tasks. LASER-KV maintains stable performance, achieving superior accuracies by a margin of upto 10% at 128k. These findings challenge the prevailing assumption that attention scores alone are a sufficient proxy for token utility.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 7 of its 14 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

7 of 14 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference11 Mar
  2. 2024Star Attention: Efficient LLM Inference over Long Sequences26 Nov
  3. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun

Other references · 4

  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  2. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  3. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
  4. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 192

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compressioncited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2024KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head
  2. 2025Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques
  3. 2026LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
  4. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
  5. 2026KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models
  6. 2026EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
  7. 2026KV-Kaizen: Learning Context-Adaptive Cache Compression Choices
  8. 2024ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty