mlsanityData to 14 Oct 2026

KQ-SVD: Compressing the KV Cache with Provable Guarantees on Attention Fidelity

arXiv 2512.05916 ↗First posted 5 Dec 2025cs.LGcited by 10

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

The Key-Value (KV) cache is central to the efficiency of transformer-based large language models (LLMs), storing previously computed vectors to accelerate inference. Yet, as sequence length and batch size grow, the cache becomes a major memory bottleneck. Prior compression methods typically apply low-rank decomposition to keys alone or attempt to jointly embed queries and keys, but both approaches neglect that attention fundamentally depends on their inner products. In this work, we prove that such strategies are suboptimal for approximating the attention matrix. We introduce KQ-SVD, a simple and computationally efficient method that directly performs an optimal low-rank decomposition of the attention matrix via a closed-form solution. By targeting the true source of redundancy, KQ-SVD preserves attention outputs with higher fidelity under compression. Extensive evaluations on LLaMA and Mistral models demonstrate that our approach consistently delivers superior projection quality.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 35 references that are in the map. First posted on arXiv 5 Dec 2025.

Builds on

23 of 35 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference28 Oct
  2. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy4 Oct
  3. 2024Eigen Attention: Attention in Low-Rank Space for KV Cache Compression10 Aug
  4. 2024Palu: Compressing KV-Cache with Low-Rank Projection30 Jul
  5. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption25 Jul
  6. 2024Effectively Compress KV Heads for LLM11 Jun

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  11. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  12. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 169
5 more
  1. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 89
  2. 2024Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysiscited by 44
  3. 2021Sparse Attention with Linear Unitscited by 41
  4. 2022Beyond Adult and COMPAS: Fairness in Multi-Class Predictioncited by 16
  5. 2025SQuat: Subspace-orthogonal KV Cache Quantizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Don't be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifoldcited by 3
  2. 2026WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantizationcited by 3
  3. 2026Attention-Weighted Value Projection for KV-Cache Compressioncited by 2
  4. 2026LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decodingcited by 1
  5. 2026KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocationcited by 1
  6. 2026From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Servingcited by 0
  7. 2026A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocationcited by 0
  8. 2026AttSVD:Prompt-Adaptive Low-Rank KV Cache Compression via Attention-Guided SVDcited by 0
  9. 2026Freeze the Decoder, Heal the Encoder: Parameter-Efficient Adaptation for SVD-Based KV-Cache Compressioncited by 0
  10. 2026Joint Subspace Sparse Attention for Efficient Long-Context Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
  2. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression
  3. 2025SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention
  4. 2025SALS: Sparse Attention in Latent Space for KV cache Compression
  5. 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
  6. 2025SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
  7. 2025KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
  8. 2026DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation