mlsanityData to 7 Oct 2026

Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention

arXiv 2610.03135 ↗First posted 2 Oct 2026cs.LGcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Serving long-context autoregressive language models is constrained by the key-value (KV) cache. Most dynamic eviction methods score token importance per query head and choose tokens independently. This fits poorly with grouped-query attention (GQA), where several query heads share one physical KV buffer: divergent per-head selections force the serving engine to retain the union of their choices - inflating the cache by up to the group ratio r - while arithmetic mean pooling dilutes the specialized retrieval heads that carry factual recall. We introduce Page-EntroKV, a formal framework for KV-cache eviction operating at the granularity GQA serving actually allocates. Heads within each physical group are pooled by weights derived from sink-isolated collision (Renyi-2) entropy - one inner product per head, computed once at prefill with no calibration - so sink heads cannot masquerade as retrieval heads. Pooled scores are projected onto PagedAttention page frames, and eviction executes at the hardware tuple (layer, group, page). We formalize the union overhead ratio (UOR) and intra-group disagreement, prove an exact identity linking them for two-head groups alongside two-sided bounds at every group ratio, prove strict budget preservation and a finite-context needle-retention bound that arithmetic mean pooling provably violates, and give exact per-layer page accounting. On a pilot architecture (Qwen2.5-1.5B-Instruct, r=6), head-independent replay over 2,240 group measurements yields union overhead up to 4.75x at a 2% budget, while Page-EntroKV holds UOR exactly 1.000; sink isolation removes a 13x sink masquerade; needle recall is 100% versus 0% for mean pooling at a 20% budget; retained cardinality is exact for every page size; and QA and code tasks remain solvable at 20% retention.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 24 of its 29 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

24 of 29 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
  2. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  3. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  4. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  6. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  8. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  10. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  12. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
6 more
  1. 2019Are Sixteen Heads Really Better than One?cited by 803
  2. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  3. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  4. 2022Efficiently Scaling Transformer Inferencecited by 529
  5. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  6. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 192

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026KV2: A Self-Refining KV Cache
  2. 2026Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
  3. 2026Trust the Mass: Forced Weights in KV-Cache Eviction
  4. 2025CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation
  5. 2025REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression
  6. 2025CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences
  7. 2026Randomization Boosts KV Caching, Learning Balances Query Load: A Joint Perspective
  8. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time