mlsanityData to 14 Oct 2026

ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration

arXiv 2505.24357 ↗First posted 30 May 2025cs.LG cs.AIcited by 16

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable performance, but their long-context reasoning remains constrained by the excessive memory required for the Key-Value (KV) cache. This makes KV cache compression a critical step toward efficient long-context inference. Recent methods have explored low-rank techniques to reduce the hidden size of the KV cache. However, they neglect the distinct roles and varying importance of Keys and Values, leading to significant performance drops under high compression. To address this, we propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values. For Keys, we propose Head-wise Similarity aware Reordering (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation via grouped SVD. For Values, we propose Offline Value Calibration (OVC), which efficiently calibrates the value projection matrix using calibration data without training, ensuring an accurate representation of contextual information. Extensive experiments show that ReCalKV consistently outperforms existing low-rank compression methods, achieving high compression ratios with minimal performance loss. The code and models will be available at:https://github.com/XIANGLONGYAN/ReCalKV.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 42 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

35 of 42 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct
  2. 2024MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection16 Oct
  3. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy4 Oct
  4. 2024Eigen Attention: Attention in Low-Rank Space for KV Cache Compression10 Aug
  5. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Models23 May
  6. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  7. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference14 Feb
  8. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  12. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
15 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  5. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  6. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  7. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  9. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  10. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 243
  11. 2017Benchmarking Denoising Algorithms with Real Photographscited by 209
  12. 2022Language model compression with weighted low-rank factorizationcited by 195
  13. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 169
  14. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  15. 2025AdaSVD: Adaptive Singular Value Decomposition for Large Language Modelscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

  1. 2026STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Controlcited by 11
  2. 2026OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantizationcited by 9
  3. 2025OjaKV: Context-Aware Online Low-Rank KV Cache Compressioncited by 7
  4. 2025XQuant: Breaking the Memory Wall for LLM Inference with KV Cache Rematerializationcited by 4
  5. 2025FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image Editingcited by 4
  6. 2026Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimizationcited by 4
  7. 2026C2KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inferencecited by 4
  8. 2026RAP: KV-Cache Compression via RoPE-Aligned Pruningcited by 3
  9. 2026Attention-Weighted Value Projection for KV-Cache Compressioncited by 2
  10. 2026Joint Encoding of KV-Cache Blocks for Scalable LLM Servingcited by 1
  11. 2026EinSort: Sorting is All We Need for Tensorizing LLMcited by 0
  12. 2026A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocationcited by 0
4 more
  1. 2026DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocationcited by 0
  2. 2026S4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Cachingcited by 0
  3. 2026AttSVD:Prompt-Adaptive Low-Rank KV Cache Compression via Attention-Guided SVDcited by 0
  4. 2026Freeze the Decoder, Heal the Encoder: Parameter-Efficient Adaptation for SVD-Based KV-Cache Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
  2. 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
  3. 2026MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference
  4. 2025LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
  5. 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
  6. 2026One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
  7. 2025KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
  8. 2025Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques