mlsanityData to 7 Oct 2026

Ablation, Statistical Inference, and Validation for KV-Cache Compression

arXiv 2607.09683 ↗First posted 14 Jun 2026cs.LG cs.AI cs.ITcited by 0

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

This study systematically compares Turbo-Quant and SpectralQuant KV-cache compression, evaluating non-dominated schemes, including WHT rotation with Beta Lloyd-Max and QJL, through a statistical validation methodology that separates systematic codec differences from implementation variance. Key findings reveal that while eigenbasis-based methods fail on heavy-tailed data due to covariance instability, they excel in structured regimes, with the effective semantic dimension (deff) adapting to calibration budgets rather than true data rank. (this is an abstract of the abstract thank you )

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 5 of its 7 references that are in the map. First posted on arXiv 14 Jun 2026.

Builds on

5 of 7 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Statistical Inference and Quality Measures of KV Cache Quantisations Inspired by TurboQuant27 Apr
  2. 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead5 Jun
  3. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb
  4. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization31 Jan

Other references · 1

  1. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026D-Quant: Driftable Entropy Coding for KV Cache Quantization
  2. 2026The risk of KV cache compression
  3. 2026TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
  4. 2026OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
  5. 2026KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs
  6. 2026Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit
  7. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
  8. 2024Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression