# KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correction

arXiv: https://arxiv.org/abs/2510.05373 · HTML: https://mlsanity.com/paper/2510.05373/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 34 of its 41 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Placed in

- Efficient LLM inference and serving / [KV cache quantization](https://mlsanity.com/inference/kv-cache-quantization/)

## Builds on (most cited first, 30 of 34)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache](https://mlsanity.com/paper/2402.02750/) (2024)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization](https://mlsanity.com/paper/2401.18079/) (2024)
- [Gated Linear Attention Transformers with Hardware-Efficient Training](https://mlsanity.com/paper/2312.06635/) (2023)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [Simple linear attention language models balance the recall-throughput tradeoff](https://mlsanity.com/paper/2402.18668/) (2024)
- [RedPajama: an Open Dataset for Training Large Language Models](https://mlsanity.com/paper/2411.12372/) (2024)
- [GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM](https://mlsanity.com/paper/2403.05527/) (2024)
- [No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization](https://mlsanity.com/paper/2402.18096/) (2024)
- [ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification](https://mlsanity.com/paper/2405.14256/) (2024)
- [Eigen Attention: Attention in Low-Rank Space for KV Cache Compression](https://mlsanity.com/paper/2408.05646/) (2024)
- [WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More](https://mlsanity.com/paper/2402.12065/) (2024)
- [QAQ: Quality Adaptive Quantization for LLM KV Cache](https://mlsanity.com/paper/2403.04643/) (2024)
- [QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead](https://mlsanity.com/paper/2406.03482/) (2024)
- [SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models](https://mlsanity.com/paper/2405.06219/) (2024)
- [ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals](https://mlsanity.com/paper/2412.14363/) (2024)
- [Liger: Linearizing Large Language Models to Gated Recurrent Structures](https://mlsanity.com/paper/2503.01496/) (2025)

## Built on by (most cited first, 10 of 10)

- [OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization](https://mlsanity.com/paper/2605.17757/) (2026)
- [PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference](https://mlsanity.com/paper/2604.24971/) (2026)
- [SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving](https://mlsanity.com/paper/2605.01708/) (2026)
- [Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion](https://mlsanity.com/paper/2605.26266/) (2026)
- [AnchorKV: Anchor-Residual KV Cache Compression](https://mlsanity.com/paper/2608.02901/) (2026)
- [Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression](https://mlsanity.com/paper/2605.27646/) (2026)
- [Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach](https://mlsanity.com/paper/2606.21448/) (2026)
- [Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches](https://mlsanity.com/paper/2607.20538/) (2026)
- [Transforms for LLM Quantization: The Great Inversion and Format Co-Design](https://mlsanity.com/paper/2608.25188/) (2026)
- [Quality Recovery for Quantized KV Caches via Low-Rank Attention Adaptation](https://mlsanity.com/paper/2609.04263/) (2026)

## Abstract (arXiv)

Quantizing the key-value (KV) cache is a promising strategy for improving the inference efficiency of large language models (LLMs). However, aggressive quantization to very low precision (e.g., 2 bits) introduces significant errors in the stored key and value tensors, which propagate through the dot-product attention mechanism and ultimately degrade generation quality. To address this, we propose KVLinC, a framework to mitigate attention errors introduced by KV cache quantization in the extreme low-precision regime. KVLinC combines a Hadamard rotation, which reduces quantization error in values, with lightweight linear correction adapters that explicitly compensate for errors introduced by quantized keys. Across extensive evaluations on the LLaMA, Qwen2.5, and Qwen3 model families, KVLinC consistently matches or surpasses strong baselines while achieving higher KV-cache compression. Furthermore, we implement a custom attention kernel that results in upto 2.55x faster inference compared to Flash Attention baseline, enabling efficient long-context LLM inference.