# KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs

arXiv: https://arxiv.org/abs/2508.04257 · HTML: https://mlsanity.com/paper/2508.04257/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving.
- First posted on arXiv 6 Aug 2025.

## Placed in

- Efficient LLM inference and serving / [KV cache quantization](https://mlsanity.com/inference/kv-cache-quantization/)

## Built on by (most cited first, 20 of 20)

- [The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks](https://mlsanity.com/paper/2603.05498/) (2026)
- [Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization](https://mlsanity.com/paper/2602.02958/) (2026)
- [Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers](https://mlsanity.com/paper/2510.09017/) (2025)
- [Value-Aware Stochastic KV Cache Eviction for Reasoning Models](https://mlsanity.com/paper/2606.03928/) (2026)
- [XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression](https://mlsanity.com/paper/2601.01204/) (2026)
- [Block Cascading: Training Free Acceleration of Block-Causal Video Models](https://mlsanity.com/paper/2511.20426/) (2025)
- [Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse](https://mlsanity.com/paper/2602.01203/) (2026)
- [OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond](https://mlsanity.com/paper/2605.19660/) (2026)
- [Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sink](https://mlsanity.com/paper/2602.05228/) (2026)
- [SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipelining](https://mlsanity.com/paper/2602.10718/) (2026)
- [When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models](https://mlsanity.com/paper/2604.03316/) (2026)
- [InnerQ: Hardware-Aware Tuning-Free Quantization of KV Cache for Large Language Models](https://mlsanity.com/paper/2602.23200/) (2026)
- [Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers](https://mlsanity.com/paper/2603.17771/) (2026)
- [Attention Sinks and Outliers in Attention Residuals](https://mlsanity.com/paper/2605.17887/) (2026)
- [Inference Time Optimization with Confidence Dynamics](https://mlsanity.com/paper/2605.25244/) (2026)
- [Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus](https://mlsanity.com/paper/2608.12149/) (2026)
- [RoPE-Aware Bit Allocation for KV-Cache Quantization](https://mlsanity.com/paper/2606.24033/) (2026)
- [XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression](https://mlsanity.com/paper/2602.21780/) (2026)
- [KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization](https://mlsanity.com/paper/2610.14448/) (2026)
- [Before Attention Sinks Form: Hidden-State Geometry Reveals and Modulates Their Formation](https://mlsanity.com/paper/2610.16745/) (2026)

## Abstract (arXiv)

Key-Value (KV) cache quantization has become a widely adopted optimization technique for efficient large language models (LLMs) inference by reducing KV cache memory usage and mitigating memory-bound constraints. Recent studies have emphasized the importance of preserving the original precision of KVs for the first few tokens to ensure the protection of attention sinks. While this approach has proven effective in mitigating performance degradation, its underlying principles remain insufficiently understood. Moreover, it fails to address the recent discovery that attention sinks can emerge beyond the initial token positions. In this work, we elucidate the underlying mechanisms of attention sinks during inference by examining their role in the cross-layer evolution of extreme activation outliers. Additionally, we provide a comprehensive analysis of the interplay between attention sinks and KV cache quantization. Based on our enhanced understanding, we introduce textitKVSink, a plug-and-play method that effectively predicts sink tokens with negligible overhead, enabling more thorough preservation. Extensive experiments demonstrate that KVSink outperforms the existing Preserve-First-N (PFN) strategy, offering more effective preservation of attention sinks during KV cache quantization. Moreover, when applied to the well-established KVQuant method, KVSink further improves perplexity (PPL) and reduces reliance on 16-bit numerical outliers.