# Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics

arXiv: https://arxiv.org/abs/2603.01426 · HTML: https://mlsanity.com/paper/2603.01426/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- First posted on arXiv 2 Mar 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Built on by (most cited first, 6 of 6)

- [How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers](https://mlsanity.com/paper/2604.17935/) (2026)
- [KV Cache Offloading for Context-Intensive Tasks](https://mlsanity.com/paper/2604.08426/) (2026)
- [Transactional Attention: Semantic Sponsorship for KV-Cache Retention](https://mlsanity.com/paper/2604.11288/) (2026)
- [Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation](https://mlsanity.com/paper/2608.01676/) (2026)
- [Pixel Decodability Is Not a Compression Signal: Causally Evaluating Importance Proxies for Visual KV-Cache Eviction](https://mlsanity.com/paper/2609.13012/) (2026)
- [PAGE: Partition-Aware Gated KV-Cache Eviction](https://mlsanity.com/paper/2609.22157/) (2026)

## Abstract (arXiv)

As context windows in LLMs scale to 100K+ tokens, the key-value (KV) cache becomes the dominant memory bottleneck, with recent methods claiming 80-90% savings and minimal benchmark degradation. We argue these evaluations miss a structural issue: attention is not just storage but routing, and retaining KV pairs does not guarantee semantic accessibility. We propose a physics-inspired view of KV compression as a controlled perturbation of token-level routing, distinguishing retention, accessibility, and utilization. Using synthetic tasks probing multi-entity tracking, disambiguation, coreference, and multi-hop reasoning, we find that moderate compression degrades internal representations with little accuracy loss, revealing redundancy; all models exhibit a sharp hallucination safety cliff near 90% compression, correlated with spikes in Global Eviction Ratio (GER), suggesting a phase transition in semantic reachability; and architectures differ in routing dynamics, with LLaMA showing early consensus and late diversification, and Qwen showing funnel-like late convergence, leading to distinct resilience profiles. Beyond erasure, we identify representational rigidity, where excessive head-level consensus collapses routing flexibility despite token survival. These results suggest sparse token-route structures govern compression tolerance, reframing KV compression as a structural probe of attention geometry and linking long-context scalability to sparsity and the lottery ticket hypothesis in self-attention.