# KV cache optimization

Area: [Efficient LLM inference and serving](https://mlsanity.com/inference/index.md). HTML: https://mlsanity.com/inference/kv-cache-optimization/

## Key facts

- KV cache optimization: 708 papers on arXiv since 2022 (data through 7 Oct 2026).
- 319 of its 708 mapped papers (45%) cite “Efficient Memory Management for Large Language Model Serving with PagedAttention” (2023, arXiv 2309.06180).
- 313 of its 708 mapped papers (44%) cite “SnapKV: LLM Knows What You are Looking for Before Generation” (2024, arXiv 2404.14469).
- 45 new papers 24 Sep–7 Oct.

## Most cited here

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023), cited by 319 papers here
- [SnapKV: LLM Knows What You are Looking for Before Generation](https://mlsanity.com/paper/2404.14469/) (2024), cited by 313 papers here
- [H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models](https://mlsanity.com/paper/2306.14048/) (2023), cited by 290 papers here
- [Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time](https://mlsanity.com/paper/2305.17118/) (2023), cited by 182 papers here
- [Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference](https://mlsanity.com/paper/2406.10774/) (2024), cited by 180 papers here
- [PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling](https://mlsanity.com/paper/2406.02069/) (2024), cited by 176 papers here
- [Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs](https://mlsanity.com/paper/2310.01801/) (2023), cited by 172 papers here
- [Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference](https://mlsanity.com/paper/2407.11550/) (2024), cited by 119 papers here
- [InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management](https://mlsanity.com/paper/2406.19707/) (2024), cited by 99 papers here
- [DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads](https://mlsanity.com/paper/2410.10819/) (2024), cited by 98 papers here

## New (24 Sep–7 Oct)

- [A Self-Pruning Transformer: Extreme KV-Cache Compression with Universal Attention](https://mlsanity.com/paper/2610.09051/) (2026-10-06)
- [SPIN: Shadow Predictive Indexer for Sparse Attention](https://mlsanity.com/paper/2610.09025/) (2026-10-06)
- [Enabling Dynamic Computation in Looped LMs](https://mlsanity.com/paper/2610.09013/) (2026-10-06)
- [Monte Carlo Estimation for KV Cache Eviction](https://mlsanity.com/paper/2610.07643/) (2026-10-06)
- [OVAL: Output-Aware Local Page Bases for KV Cache Retrieval](https://mlsanity.com/paper/2610.06686/) (2026-10-05)
- [Behavior-Preserving KV Cache Compression](https://mlsanity.com/paper/2610.06479/) (2026-10-05)
- [DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression](https://mlsanity.com/paper/2610.06286/) (2026-10-05)
- [Mask-Guided KV Cache Eviction in Block Diffusion Language Models](https://mlsanity.com/paper/2610.06996/) (2026-10-04)
- [AttSVD:Prompt-Adaptive Low-Rank KV Cache Compression via Attention-Guided SVD](https://mlsanity.com/paper/2610.06927/) (2026-10-03)
- [MOIRA: Mass-Oriented Indexing with Ragged Attention for Long-Context Decoding](https://mlsanity.com/paper/2610.04313/) (2026-10-03)
- [KV2: A Self-Refining KV Cache](https://mlsanity.com/paper/2610.03198/) (2026-10-02)
- [Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention](https://mlsanity.com/paper/2610.03135/) (2026-10-02)
- [SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention](https://mlsanity.com/paper/2610.02953/) (2026-10-02)
- [iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD](https://mlsanity.com/paper/2610.02815/) (2026-10-02)
- [WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Minds](https://mlsanity.com/paper/2610.02713/) (2026-10-02)
- [CommunityKV: Efficient Long-Context Decoding via Graph Partitioning](https://mlsanity.com/paper/2610.00418/) (2026-09-30)
- [EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction](https://mlsanity.com/paper/2610.00412/) (2026-09-30)
- [PatchKV: Weight-Space Compensation of KV Cache](https://mlsanity.com/paper/2609.39329/) (2026-09-30)
- [SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs](https://mlsanity.com/paper/2609.38830/) (2026-09-30)
- [KV-Kaizen: Learning Context-Adaptive Cache Compression Choices](https://mlsanity.com/paper/2609.37988/) (2026-09-29)
- [ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction](https://mlsanity.com/paper/2609.36835/) (2026-09-29)
- [ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation](https://mlsanity.com/paper/2609.36722/) (2026-09-29)
- [CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache](https://mlsanity.com/paper/2610.02235/) (2026-09-28)
- [Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression](https://mlsanity.com/paper/2609.36322/) (2026-09-28)
- [Cartridges++: KV Cache Compression without Off-Context Derailment](https://mlsanity.com/paper/2609.35621/) (2026-09-28)
- [CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion](https://mlsanity.com/paper/2609.35139/) (2026-09-28)
- [TempoKV: Timely Staging of LLM KV Caches for Memory-Semantic Flash](https://mlsanity.com/paper/2609.35065/) (2026-09-28)
- [PulseInfer: I/O-Centric Sparse KV Cache Offloading for Efficient Long-Context LLM Decoding](https://mlsanity.com/paper/2609.34555/) (2026-09-28)
- [Thinking Outside the Box: Retention and Transmission of Information in Sliding-Window KV Inference](https://mlsanity.com/paper/2609.34049/) (2026-09-28)
- [Where Activation Sparsity and KV-Cache Sparsity Cross in LLM Decoding](https://mlsanity.com/paper/2609.33889/) (2026-09-27)