mlsanityData to 7 Oct 2026

VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization

arXiv 2603.16435 ↗First posted 17 Mar 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

The growing context length of Large Language Models (LLMs) enlarges the Key-Value (KV) cache, limiting deployment in resource-limited environments. Prior training-free approaches for KV cache compression typically rely on low-rank approximation or scalar quantization, which fail to simultaneously achieve high compression ratios and high reconstruction fidelity. We propose VQKV, a novel, training-free method introducing vector quantization (VQ) to obtain highly compressed KV representations while preserving high model fidelity, allowing for the representation of thousands of floating-point values with just a few integer indices. As a result, VQKV achieves an 82.8% compression ratio on LLaMA3.1-8B while retaining 98.6% of the baseline performance on LongBench and enabling 4.3x longer generation length on the same memory footprint.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

30 of 39 references.

References · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  9. 2017Billion-scale similarity search with GPUscited by 2,477
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  11. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  12. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
18 more
  1. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  2. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  3. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  4. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  6. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  7. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  8. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  9. 2024The Faiss librarycited by 370
  10. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 156
  11. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
  12. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
  13. 2024Addressing Representation Collapse in Vector Quantized Models with One Linear Layercited by 77
  14. 2025VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizerscited by 67
  15. 2025VQ-LLM: High-performance Code Generation for Vector Quantization Augmented LLM Inferencecited by 12
  16. 2026Next Concept Prediction in Discrete Latent Space Leads to Stronger Language Modelscited by 8
  17. 2025Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cachecited by 5
  18. 2024CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarioscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Hurwitz Quaternion Multiplicative Quantization for KV Cache Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models
  2. 2025PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
  3. 2026Tailoring the Quantization Space for 1-Bit KV Cache Compression
  4. 2025CommVQ: Commutative Vector Quantization for KV Cache Compression
  5. 2024SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
  6. 2025NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics
  7. 2025SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention
  8. 2025AnTKV: Anchor Token-Aware Sub-Bit Vector Quantization for KV Cache in Large Language Models