mlsanityData to 7 Oct 2026

KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs

arXiv 2602.05929 ↗First posted 5 Feb 2026cs.CLcited by 5

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Large language models rely on kv-caches to avoid redundant computation during autoregressive decoding, but as context length grows, reading and writing the cache can quickly saturate GPU memory bandwidth. Recent work has explored KV-cache compression, yet most approaches neglect the data-dependent nature of kv-caches and their variation across layers. We introduce KV-CoRE KV-cache Compressibility by Rank Evaluation), an SVD-based method for quantifying the data-dependent low-rank compressibility of kv-caches. KV-CoRE computes the optimal low-rank approximation under the Frobenius norm and, being gradient-free and incremental, enables efficient dataset-level, layer-wise evaluation. Using this method, we analyze multiple models and datasets spanning five English domains and sixteen languages, uncovering systematic patterns that link compressibility to model architecture, training data, and language coverage. As part of this analysis, we employ the Normalized Effective Rank as a metric of compressibility and show that it correlates strongly with performance degradation under compression. Our study establishes a principled evaluation framework and the first large-scale benchmark of kv-cache compressibility in LLMs, offering insights for dynamic, data-aware compression and data-centric model development.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 39 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

22 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs20 Feb
  2. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption25 Jul

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  5. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  6. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
  7. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  8. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 231
  9. 2022Language model compression with weighted low-rank factorizationcited by 192
  10. 2022Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapsecited by 136
  11. 2021Scatterbrain: Unifying Sparse and Low-rank Attention Approximationcited by 128
  12. 2023LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximationcited by 103
8 more
  1. 2020Low-Rank Bottleneck in Multi-head Attention Modelscited by 88
  2. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 87
  3. 2023Transformers learn through gradual rank increasecited by 52
  4. 2025AI Flow: Perspectives, Scenarios, and Approachescited by 44
  5. 2024Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptationcited by 26
  6. 2025VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understandingcited by 13
  7. 2025HPU: High-Bandwidth Processing Unit for Scalable, Cost-effective LLM Inference via GPU Co-processingcited by 5
  8. 2023A Survey of Historical Learning: Learning Models with Learning Historycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compressioncited by 2
  2. 2026KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Modelscited by 1
  3. 2026Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantizationcited by 0
  4. 2026iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVDcited by 0
  5. 2026Freeze the Decoder, Heal the Encoder: Parameter-Efficient Adaptation for SVD-Based KV-Cache Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
  2. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
  3. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
  4. 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
  5. 2026EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
  6. 2025KV Cache Transform Coding for Compact Storage in LLM Inference
  7. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  8. 2025PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression