mlsanityData to 14 Oct 2026

EvolKV: Evolutionary KV Cache Compression for LLM Inference

arXiv 2509.08315 ↗First posted 10 Sep 2025cs.LG cs.CL cs.NEcited by 10

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Existing key-value (KV) cache compression methods typically rely on heuristics, such as uniform cache allocation across layers or static eviction policies, however, they ignore the critical interplays among layer-specific feature patterns and task performance, which can lead to degraded generalization. In this paper, we propose EvolKV, an adaptive framework for layer-wise, task-driven KV cache compression that jointly optimizes the memory efficiency and task performance. By reformulating cache allocation as a multi-objective optimization problem, EvolKV leverages evolutionary search to dynamically configure layer budgets while directly maximizing downstream performance. Extensive experiments on 11 tasks demonstrate that our approach outperforms all baseline methods across a wide range of KV cache budgets on long-context tasks and surpasses heuristic baselines by up to 7 percentage points on GSM8K. Notably, EvolKV achieves superior performance over the full KV cache setting on code completion while utilizing only 1.5% of the original budget, suggesting the untapped potential in learned compression strategies for KV cache budget allocation.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 46 references that are in the map. First posted on arXiv 10 Sep 2025.

Builds on

30 of 46 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption25 Jul
  2. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  3. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference21 May
  4. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  5. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  9. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  10. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  11. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  12. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
13 more
  1. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  2. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  3. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  4. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  5. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  6. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
  7. 2021Efficient Attentions for Long Document Summarizationcited by 345
  8. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  9. 2023LongCoder: A Long-Range Pre-trained Language Model for Code Completioncited by 112
  10. 2020Rethinking the Value of Transformer Componentscited by 25
  11. 2023Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Textcited by 19
  12. 2022Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewardscited by 14
  13. 2024Autoregressive Pre-Training on Pixels and Textscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Servingcited by 10
  2. 2026Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimizationcited by 4
  3. 2026Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inferencecited by 3
  4. 2025SR-KI: Scalable and Real-Time Knowledge Integration into LLMs via Supervised Attentioncited by 2
  5. 2026CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inferencecited by 1
  6. 2026DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compressioncited by 1
  7. 2026FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inferencecited by 0
  8. 2026SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensationcited by 0
  9. 2026Discovering KV Cache Eviction Policies via LLM-Guided Program Evolutioncited by 0
  10. 2026What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
  2. 2024MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache
  3. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
  4. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs
  5. 2026LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
  6. 2025ReFreeKV: Towards Threshold-Free KV Cache Compression
  7. 2026Behavior-Preserving KV Cache Compression
  8. 2025BaKlaVa -- Budgeted Allocation of KV cache for Long-context Inference