mlsanityData to 7 Oct 2026

KV cache quantization

122 papers4 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: quantization, cache, bit, memory, cache quantization

Key facts. KV cache quantization: 122 papers on arXiv since 2023 (data through 7 Oct 2026). 75 of its 122 mapped papers (61%) cite “KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization” (2024, arXiv 2401.18079). 74 of its 122 mapped papers (61%) cite “KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache” (2024, arXiv 2402.02750). 8 new papers 24 Sep–7 Oct.

What’s new

24 Sep – 7 Oct 2026

8 new papers, 24 Sep – 7 Oct; 4 this week.

  1. This week · 4
  2. ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
  3. Dual-QK: Sharp Queries and Flat Keys for Prunable 2-bit KV Caches
  4. Few Bits, One Law: Toward W2A4KV2
  5. AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation
  6. Earlier · 4
  7. Tailoring the Quantization Space for 1-Bit KV Cache Compression
  8. STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
  9. WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms
  10. Behavioral Capacity Certificates for Quantized Language Models also fits: Efficient reasoning and agent inference

Foundations

Most cited by papers here

  1. KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization2024cited by 75 here
  2. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache2024cited by 74 here
  3. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs2024cited by 51 here
  4. SpinQuant: LLM quantization with learned rotations2024cited by 30 here
  5. Atom: Low-bit Quantization for Efficient and Accurate LLM Serving2023cited by 29 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 46 here
  2. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 29 here
  3. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 75 here
  4. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 74 here
  5. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 51 here
  6. 2024SpinQuant: LLM quantization with learned rotationscited by 30 here
  7. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotationscited by 15 here
17 more it builds on, not in this chain
  1. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 25 here
  2. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 41 here
  3. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 19 here
  4. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 8 here
  5. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantizationcited by 25 here
  6. 2024QAQ: Quality Adaptive Quantization for LLM KV Cachecited by 19 here
  7. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 21 here
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 29 here
  9. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantizationcited by 16 here
  10. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 29 here
  11. 2024SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Modelscited by 14 here
  12. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 20 here
  13. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 17 here
  14. 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overheadcited by 16 here
  15. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 10 here
  16. 2024Palu: Compressing KV-Cache with Low-Rank Projectioncited by 13 here
  17. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 12 here

The 24 most cited of 234.

Browse

30 of 122, ranked 6–35

  1. 6QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving202429
  2. 7ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification202420
  3. 8QAQ: Quality Adaptive Quantization for LLM KV Cache202419
  4. 9KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization202416
  5. 10QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead202416
  6. 11RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations202515
  7. 12SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models202414
  8. 13KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference202512
  9. 14PolarQuant: Quantizing KV Caches with Polar Transformation20258
  10. 15PQCache: Product Quantization-based KVCache for Long Context LLM Inference20248
All 30
  1. 16Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost20257
  2. 17AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations20247
  3. 18CommVQ: Commutative Vector Quantization for KV Cache Compression20256
  4. 19Quantize What Counts: More for Keys, Less for Values20255
  5. 20Accurate KV Cache Quantization with Outlier Tokens Tracing20255
  6. 21PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs20255
  7. 22KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correction20254
  8. 23KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks20263
  9. 24KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs20253
  10. 25MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning20253
  11. 26Residual vector quantization for KV cache compression in large language model20243
  12. 27IsoQuant: Hardware-Aligned SO(4) Isoclinic Rotations for LLM KV Cache Compression20262
  13. 28FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression20262
  14. 29OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization20262
  15. 30OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization20262
  16. 31PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration20252
  17. 32Cocktail: Chunk-Adaptive Mixed-Precision Quantization for Long-Context LLM Inference20252
  18. 33SQuat: Subspace-orthogonal KV Cache Quantization20252
  19. 34KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache20252
  20. 35NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache20252