mlsanityData to 14 Oct 2026

KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correction

arXiv 2510.05373 ↗First posted 6 Oct 2025cs.LGcited by 10

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Quantizing the key-value (KV) cache is a promising strategy for improving the inference efficiency of large language models (LLMs). However, aggressive quantization to very low precision (e.g., 2 bits) introduces significant errors in the stored key and value tensors, which propagate through the dot-product attention mechanism and ultimately degrade generation quality. To address this, we propose KVLinC, a framework to mitigate attention errors introduced by KV cache quantization in the extreme low-precision regime. KVLinC combines a Hadamard rotation, which reduces quantization error in values, with lightweight linear correction adapters that explicitly compensate for errors introduced by quantized keys. Across extensive evaluations on the LLaMA, Qwen2.5, and Qwen3 model families, KVLinC consistently matches or surpasses strong baselines while achieving higher KV-cache compression. Furthermore, we implement a custom attention kernel that results in upto 2.55x faster inference compared to Flash Attention baseline, enabling efficient long-context LLM inference.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 34 of its 41 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

34 of 41 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025PolarQuant: Quantizing KV Caches with Polar Transformation4 Feb
  2. 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead5 Jun
  3. 2024SpinQuant: LLM quantization with learned rotations26 May
  4. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification23 May
  5. 2024SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models10 May
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar
  7. 2024QAQ: Quality Adaptive Quantization for LLM KV Cache7 Mar
  8. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb
  9. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization31 Jan

Other references · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  9. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  10. 2020Rethinking Attention with Performerscited by 1,601
  11. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  12. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
13 more
  1. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  2. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 471
  3. 2024Simple linear attention language models balance the recall-throughput tradeoffcited by 208
  4. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
  5. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 158
  6. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantizationcited by 101
  7. 2024Eigen Attention: Attention in Low-Rank Space for KV Cache Compressioncited by 72
  8. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  9. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 36
  10. 2025Liger: Linearizing Large Language Models to Gated Recurrent Structurescited by 33
  11. 2024LoLCATs: On Low-Rank Linearizing of Large Language Modelscited by 27
  12. 2025LoLA: Low-Rank Linear Attention With Sparse Cachingcited by 12
  13. 2023QBitOpt: Fast and Accurate Bitwidth Reallocation during Trainingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantizationcited by 9
  2. 2026PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inferencecited by 3
  3. 2026SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Servingcited by 3
  4. 2026Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusioncited by 2
  5. 2026AnchorKV: Anchor-Residual KV Cache Compressioncited by 1
  6. 2026Hurwitz Quaternion Multiplicative Quantization for KV Cache Compressioncited by 0
  7. 2026Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approachcited by 0
  8. 2026Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Cachescited by 0
  9. 2026Transforms for LLM Quantization: The Great Inversion and Format Co-Designcited by 0
  10. 2026Quality Recovery for Quantized KV Caches via Low-Rank Attention Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
  2. 2025KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache
  3. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
  4. 2025NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics
  5. 2025VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
  6. 2026High-accuracy Low-Bit KV-Cache Quantization via Local Distribution Restoration
  7. 2025PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs
  8. 2025KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference