# eOptShrinkQ: Near-Lossless KV Cache Compression Through Optimal Spectral Denoising and Quantization

arXiv: https://arxiv.org/abs/2605.02905 · HTML: https://mlsanity.com/paper/2605.02905/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [SnapKV: LLM Knows What You are Looking for Before Generation](https://mlsanity.com/paper/2404.14469/) (2024)
- [KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache](https://mlsanity.com/paper/2402.02750/) (2024)
- [KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization](https://mlsanity.com/paper/2401.18079/) (2024)
- [PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling](https://mlsanity.com/paper/2406.02069/) (2024)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [Language model compression with weighted low-rank factorization](https://mlsanity.com/paper/2207.00112/) (2022)
- [SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models](https://mlsanity.com/paper/2411.05007/) (2024)
- [GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM](https://mlsanity.com/paper/2403.05527/) (2024)
- [TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate](https://mlsanity.com/paper/2504.19874/) (2025)
- [Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference](https://mlsanity.com/paper/2403.09636/) (2024)
- [RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2405.12497/) (2024)
- [PolarQuant: Quantizing KV Caches with Polar Transformation](https://mlsanity.com/paper/2502.02617/) (2025)
- [OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage](https://mlsanity.com/paper/1306.6042/) (2013)
- [OjaKV: Context-Aware Online Low-Rank KV Cache Compression](https://mlsanity.com/paper/2509.21623/) (2025)
- [Matrix denoising for weighted loss functions and heterogeneous signals](https://mlsanity.com/paper/1902.09474/) (2019)
- [SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention](https://mlsanity.com/paper/2502.15304/) (2025)
- [CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios](https://mlsanity.com/paper/2409.10593/) (2024)
- [Diffusion geometry approach to efficiently remove electrical stimulation artifacts in intracranial electroencephalography](https://mlsanity.com/paper/1811.08428/) (2018)
- [Recovery of the fetal electrocardiogram for morphological analysis from two trans-abdominal channels via optimal shrinkage](https://mlsanity.com/paper/1904.09525/) (2019)

## Built on by (most cited first, 2 of 2)

- [KV Cache Compression Through the Lens of Transform Coding](https://mlsanity.com/paper/2608.14191/) (2026)
- [Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization](https://mlsanity.com/paper/2607.17019/) (2026)

## Abstract (arXiv)

We show that the key-value (KV) cache in transformer attention heads admits a natural decomposition into a low-rank shared context component and a full-rank per-token residual, well described by the spiked random matrix model. This observation leads to eOptShrinkQ, a two-stage compression pipeline: optimal singular value shrinkage (eOptShrink) automatically extracts the shared structure, and the residual -- which satisfies the thin shell property with delocalized coordinates -- is quantized by TurboQuant~citepzandieh2025turboquant, a recently proposed per-vector scalar quantizer with near-optimal distortion guarantees. By restoring the isotropy that scalar quantization assumes, spectral denoising eliminates the need for both outlier handling and dedicated inner product bias correction, freeing those bits for improved reconstruction. The theoretical grounding in random matrix theory provides three guarantees: automatic rank selection via the BBP phase transition, provably near-zero inner product bias on the residual, and coordinate delocalization ensuring near-optimal quantization distortion. Experimentally, we validate eOptShrinkQ on Llama-3.1-8B and Ministral-8B across three levels: per-head MSE and inner product fidelity, where eOptShrinkQ saves nearly one bit per entry over TurboQuant at equivalent quality; end-to-end on LongBench (16 tasks), where eOptShrinkQ at ~2.2 bits per entry outperforms TurboQuant at 3.0 bits; and multi-needle retrieval, where eOptShrinkQ at 2.2 bits closely matches or exceeds uncompressed FP16, suggesting that spectral denoising can act as a beneficial regularizer for retrieval-intensive tasks.