mlsanityData to 7 Oct 2026

KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks

arXiv 2606.03458 ↗First posted 2 Jun 2026cs.LGcited by 6

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Test-time scaling is a powerful approach to obtain better reasoning in large language models, but it becomes memory-bottlenecked during long-horizon decoding, as the KV-cache grows. KV-cache quantization can help improve this, but current methods are evaluated under prefill-like settings and errors behave differently under autoregressive decoding. We show that in the latter regime, quantization errors accumulate across timesteps, driven primarily by incorrect token scales. We introduce KVarN, a calibration-free KV-cache quantizer that applies a Hadamard rotation followed by a dual-scaling variance normalization across both axes of the K and V matrices. We find that this combination fixes outlying token-scale errors and substantially reduces error accumulation over existing baselines. KVarN establishes a new state-of-theart for KV-cache quantization on generative benchmarks, including MATH500, AIME24 and HumanEval, at 2-bit precision. A vLLM implementation of the KVarN method is available at https://github.com/huawei-csl/KVarN

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 6 of its 37 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

6 of 37 references.

References · 6

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  3. 2024Phi-4 Technical Reportcited by 675
  4. 2025Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mambacited by 9
  5. 2025Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boostcited by 8
  6. 2025SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weightscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantizationcited by 3
  2. 2026Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantizationcited by 0
  3. 2026Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generationcited by 0
  4. 2026Quality Recovery for Quantized KV Caches via Low-Rank Attention Adaptationcited by 0
  5. 2026Low-Bit Recurrent States in Hybrid Language Modelscited by 0
  6. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TaDA: Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centering
  2. 2025MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning
  3. 2025KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correction
  4. 2025KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
  5. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
  6. 2025PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs
  7. 2025KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache
  8. 2026Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning