mlsanityData to 14 Oct 2026

STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

arXiv 2606.08382 ↗First posted 7 Jun 2026cs.LG cs.AIcited by 11

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Low-rank projection has emerged as a promising approach for compressing the KV cache by exploiting hidden-dimension redundancy. However, prior methods rely on fixed or heuristic rank selection and struggle to achieve aggressive compression with minimal accuracy degradation. We propose STAR-KV, an adaptive low-rank KV cache compression framework with fine-grained rank control. STAR-KV encompasses 1) a differentiable thresholding mechanism that enables optimal rank selection at both attention-head and block levels, 2) a hybrid decomposition strategy that applies different low-rank factorizations according to the sensitivity of key and value projections, and 3) a low-rank-aware mixed precision quantization that leverages data statistics for near lossless low-bit quantization. Evaluated across multiple LLMs and benchmarks, STAR-KV achieves up to 75% KV cache compression and up to 20x overall KV cache reduction when combined with quantization. Enabled by custom Triton-based GPU kernels, STAR-KV delivers up to 6.9x speedup for the attention module and 3.1x end-to-end generation throughput. Our code is publicly available at: https://github.com/PriyanshBhatnagar/STAR-KV.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 24 of its 36 references that are in the map. First posted on arXiv 7 Jun 2026.

Builds on

24 of 36 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration30 May
  2. 2025Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs20 Feb
  3. 2024MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection16 Oct
  4. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy4 Oct
  5. 2024Eigen Attention: Attention in Low-Rank Space for KV Cache Compression10 Aug
  6. 2024Palu: Compressing KV-Cache with Low-Rank Projection30 Jul
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  6. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  7. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  9. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  10. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  11. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  12. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
5 more
  1. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 198
  2. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
  3. 2024AI and Memory Wallcited by 92
  4. 2025CommVQ: Commutative Vector Quantization for KV Cache Compressioncited by 28
  5. 2025Accurate KV Cache Quantization with Outlier Tokens Tracingcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantizationcited by 3
  2. 2026Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometrycited by 1
  3. 2026KV-Kaizen: Learning Context-Adaptive Cache Compression Choicescited by 1
  4. 2026A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocationcited by 0
  5. 2026Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantizationcited by 0
  6. 2026S4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Cachingcited by 0
  7. 2026Quality Recovery for Quantized KV Caches via Low-Rank Attention Adaptationcited by 0
  8. 2026Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Managementcited by 0
  9. 2026iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVDcited by 0
  10. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0
  11. 2026BELT : BEhavior-Aligned Low-Rank KV Cache Compression with Learned Top-k Rank Allocationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation
  2. 2026KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs
  3. 2026ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression
  4. 2026TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
  5. 2025CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
  6. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification
  7. 2025ReFreeKV: Towards Threshold-Free KV Cache Compression
  8. 2026Minima-KV: Retention-Preserving KV Cache Compression with Mixed-Format Paged Attention