mlsanityData to 7 Oct 2026

G-KV: Decoding-Time KV Cache Eviction with Global Attention

arXiv 2512.00504 ↗First posted 29 Nov 2025cs.CL cs.AIcited by 7

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Recent reasoning large language models (LLMs) excel in complex tasks but encounter significant computational and memory challenges due to long sequence lengths. KV cache compression has emerged as an effective approach to greatly enhance the efficiency of reasoning. However, existing methods often focus on prompt compression or token eviction with local attention score, overlooking the long-term importance of tokens. We propose G-KV, a KV cache eviction method that employs a global scoring mechanism, combining local and historical attention scores to more accurately assess token importance. Additionally, we introduce post-training techniques, including reinforcement learning and distillation, to optimize models for compressed KV cache settings. The code of this paper is available on: https://github.com/microsoft/G-KV.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 39 of its 45 references that are in the map. First posted on arXiv 29 Nov 2025.

Builds on

39 of 45 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning20 May
  2. 2025Rethinking Reflection in Pre-Training5 Apr
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2024A Survey on Large Language Model Acceleration based on KV Cache Management27 Dec
  5. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  11. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  12. 2024Qwen2 Technical Reportcited by 2,076
22 more
  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  3. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  4. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  5. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  6. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  7. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  8. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  9. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  10. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  11. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  12. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  13. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inferencecited by 148
  14. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Modelscited by 122
  15. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 115
  16. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 89
  17. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inferencecited by 75
  18. 2025CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferencescited by 59
  19. 2023Compressed Context Memory For Online Language Model Interactioncited by 42
  20. 2024SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separatorcited by 41
  21. 2025Enhancing Efficiency and Exploration in Reinforcement Learning for LLMscited by 23
  22. 2025Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Zipage: Maintain High Request Concurrency for LLM Reasoning through Compressed PagedAttentioncited by 4
  2. 2026From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Modelscited by 3
  3. 2026HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compressioncited by 2
  4. 2026Adaptive Mass-Segmented KV Compression for Long-Context Reasoningcited by 1
  5. 2026GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Servingcited by 0
  6. 2026WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Mindscited by 0
  7. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  2. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
  3. 2026ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution
  4. 2024In-context KV-Cache Eviction for LLMs via Attention-Gate
  5. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning
  6. 2025Judge Q: Trainable Queries for Optimized Information Retention in KV Cache Eviction
  7. 2026Reformulating KV Cache Eviction Problem for Long-Context LLM Inference
  8. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction