mlsanityData to 14 Oct 2026

GraphKV: Breaking the Static Selection Paradigm with Graph-Based KV Cache Eviction

arXiv 2509.00388 ↗First posted 30 Aug 2025cs.CLcited by 6

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Efficient Key-Value (KV) cache management is essential for processing long text sequences in large language models (LLMs), where memory constraints often limit performance. Conventional KV eviction strategies, such as top-k selection based on attention scores, depend on static heuristics that fail to capture the evolving implicit dependencies among tokens during inference. To overcome this, we propose GraphKV, a graph-based framework that redefines token selection for KV cache compression. In GraphKV, tokens are modeled as nodes with importance scores, and edges represent their similarity relationships. Through a decay-signal-propagation mechanism, token importance is dynamically updated by propagating information across the graph, enabling adaptive retention of the most contextually significant tokens. GraphKV can be seamlessly utilized in existing KV cache eviction methods such as SnapKV and PyramidKV in a plug-and-play manner. Codes will be released on Github.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 35 references that are in the map. First posted on arXiv 30 Aug 2025.

Builds on

21 of 35 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation18 Dec
  2. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks11 Jul
  3. 2024Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters18 Jun
  4. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression17 Jun
  5. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  6. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference21 May
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  8. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct

Other references · 13

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  6. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  7. 2019Graph Neural Networks for Social Recommendationcited by 714
  8. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  9. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  10. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 138
  11. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  12. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 89
1 more
  1. 2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compressioncited by 5
  2. 2026SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inferencecited by 2
  3. 2026Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefillingcited by 1
  4. 2026Monte Carlo Estimation for KV Cache Evictioncited by 1
  5. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0
  6. 2026CommunityKV: Efficient Long-Context Decoding via Graph Partitioningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective
  2. 2026Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
  3. 2025LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
  4. 2024ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression
  5. 2026Behavior-Preserving KV Cache Compression
  6. 2025Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques
  7. 2026A Simple Plug-in for Improving Eviction-Based KV Cache Compression
  8. 2026Back from the Future: Key-Value Cache Management by Counter-Causal Surprise