# Statistical Inference and Quality Measures of KV Cache Quantisations Inspired by TurboQuant

arXiv: https://arxiv.org/abs/2605.08114 · HTML: https://mlsanity.com/paper/2605.08114/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 2 of its 11 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache quantization](https://mlsanity.com/inference/kv-cache-quantization/)

## Builds on (most cited first, 2 of 2)

- [TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate](https://mlsanity.com/paper/2504.19874/) (2025)
- [QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead](https://mlsanity.com/paper/2406.03482/) (2024)

## Built on by (most cited first, 2 of 2)

- [UltraQuant: 4-bit KV Caching for Context-Heavy Agents](https://mlsanity.com/paper/2606.20474/) (2026)
- [Ablation, Statistical Inference, and Validation for KV-Cache Compression](https://mlsanity.com/paper/2607.09683/) (2026)

## Abstract (arXiv)

We analyse three KV cache quantization schemes under a fair bit budget: KV (scalar MSE baseline), KQV (WHT + MSE on K; WHT + MSE + QJL on V), and QKQV (WHT + MSE + QJL on both). Starting from the Beta distribution on the hypersphere, we trace how QJL on K inflates inner product variance by π/2, which softmax amplifies nonlinearly via Jensen's inequality, and we present statistical inference and information metrics to highlight practical differences. Three empirical findings emerge. (1)~At n=4 (the practically dominant budget), KQV wins on every measure -- KL divergence, geometric K error, and 6D distance -- across all distributions and ranks tested. (2)~The K--V asymmetry is unconditional: QKQV is consistently worse than KQV in KL divergence at every budget and distribution. (3)~A budget-dependent crossover exists: QKQV achieves better geometric K reconstruction at n ∈ 2,3,5, KQV at n ∈ 4,6, invariant to rank and tail weight -- an open rate-distortion problem. KL(pref | pquant), K-only by construction, bridges K direction error to routing corruption and output collapse. We present a sufficient condition when the Jensen mechanism amplifies superlinearly through the softmax. At n ∈ 2,3,5, QKQV wins geometrically because this assumption does not bind. At n=4, elevated K error and KL divergence for QKQV strongly suggest the Jensen mechanism is the operative cause of the crossover, providing a new perspective and explanation.