mlsanityData to 14 Oct 2026

KV Cache Compression for Inference Efficiency in LLMs: A Review

arXiv 2508.06297 ↗First posted 8 Aug 2025cs.DCcited by 12

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Withtherapid advancement of large language models (LLMs), the context length for inference has been continuously increasing, leading to an exponential growth in the demand for Key-Value (KV) caching. This has resulted in a significant memory bottleneck, limiting the inference efficiency and scalability of the models. Therefore, optimizing the KV cache during inference is crucial for enhancing performance and efficiency. This review systematically examines current KV cache optimization techniques, including compression strategies such as selective token strategies, quantization, and attention compression. We evaluate the effectiveness, trade-offs, and application scenarios of these methods, providing a comprehensive analysis of their impact on memory usage and inference speed. We focus on identifying the limitations and challenges of existing methods, such as compatibility issues with different models and tasks. Additionally, this review highlights future research directions, including hybrid optimization techniques, adaptive dynamic strategies, and software-hardware co-design. These approaches aim to improve inference efficiency and promote the practical application of large language models.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 27 of its 31 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

27 of 31 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024A Survey on Large Language Model Acceleration based on KV Cache Management27 Dec
  2. 2024ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty12 Dec
  3. 2024Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries12 Dec
  4. 2024EMS: Adaptive Evict-then-Merge Strategy for Head-wise KV Cache Compression Based on Global-Local Importance11 Dec
  5. 2024XKV: Personalized KV Cache Memory Reduction for Long-Context LLM Inference8 Dec
  6. 2024Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity3 Dec
  7. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing24 Oct
  8. 2024KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head30 Sep
  9. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time7 Aug
  10. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption25 Jul
  11. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Heads22 Jul
  12. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference26 Jun
5 more
  1. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion26 May
  2. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference21 May
  3. 2024SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget7 Apr
  4. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference14 Mar
  5. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM8 Mar

Other references · 10

  1. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  2. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  3. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  4. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantizationcited by 76
  5. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCachecited by 73
  6. 2024QAQ: Quality Adaptive Quantization for LLM KV Cachecited by 66
  7. 2024ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Cachingcited by 48
  8. 2024AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurationscited by 19
  9. 2024Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQAcited by 7
  10. 2024AlignedKV: Reducing Memory Access of KV-Cache with Precision-Aligned Quantizationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 11.

12 papers.

  1. 2025TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generationcited by 5
  2. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyondcited by 5
  3. 2026Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Modelscited by 5
  4. 2026A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Designcited by 5
  5. 2026RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cachecited by 4
  6. 2026What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agentscited by 4
  7. 2026Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimizationcited by 4
  8. 2026MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inferencecited by 3
  9. 2026FlexComp: One Model for Every Ratio in Context Compressioncited by 1
  10. 2026The Global Landscape of Environmental AI Regulation: From the Cost of Reasoning to a Right to Green AIcited by 0
  11. 2026From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Servingcited by 0
  12. 2026Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  2. 2025BaKlaVa -- Budgeted Allocation of KV cache for Long-context Inference
  3. 2026Comparative Characterization of KV Cache Management Strategies for LLM Inference
  4. 2026MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference
  5. 2025Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques
  6. 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
  7. 2025KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference
  8. 2025KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference