mlsanityData to 14 Oct 2026

KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

arXiv 2609.24298 ↗First posted 21 Sep 2026cs.LG cs.AIcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

What limits KV-cache compression at extreme bit-rates? We argue that it is not the choice of compression scheme, but how its budget is allocated across attention heads. Existing methods apply rank and bit-width uniformly, ignoring that each head has a different optimal mix of rank truncation and quantization. We show that co-optimizing rank and bit-width per head, using only standard low-rank projection and scalar quantization, dominates uniform allocation, with the largest gains at low bit-rates. Our method, KV-COBRA (Co-Optimized Bit-Rank Allocation), formalizes this as a resource-allocation problem: it balances rank-truncation loss against quantization loss within each head, then redistributes budget across heads to minimize total distortion. A fused Hadamard rotation equalizes per-channel variance, and reordering the SVD basis by attention-KL importance makes the solver query-aware. The same allocator extends to joint K+V compression. On perplexity, zero-shot, and long-context benchmarks from 0.5 to 4 bits per dimension (bpd), KV-COBRA shows the smallest accuracy degradation among evaluated methods at low bpd, with no per-token overhead.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 36 of its 51 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

36 of 51 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025KQ-SVD: Compressing the KV Cache with Provable Guarantees on Attention Fidelity5 Dec
  2. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction29 May
  3. 2025Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models31 Jan
  4. 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attention21 May
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  6. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM8 Mar
  7. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 29

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2021Convex optimizationcited by 2,515
  11. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  12. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
17 more
  1. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  2. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  3. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  4. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  9. 2022Efficiently Scaling Transformer Inferencecited by 539
  10. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  11. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  12. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
  13. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 116
  14. 2021EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learningcited by 55
  15. 2024RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Searchcited by 48
  16. 2025SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attentioncited by 6
  17. 2026Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Provencited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
  2. 2026RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cache
  3. 2026ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression
  4. 2026TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
  5. 2026The risk of KV cache compression
  6. 2026Every Cache Entry Earns Its Place: Global Allocation of Resolution and Coverage for KV Cache Compression
  7. 2026BELT : BEhavior-Aligned Low-Rank KV Cache Compression with Learned Top-k Rank Allocation
  8. 2026CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache