mlsanityData to 14 Oct 2026

LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning

arXiv 2506.15969 ↗First posted 19 Jun 2025cs.LG cs.CLcited by 16

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) exhibit enhanced capabilities by Chain-of-Thought reasoning. However, the extended reasoning sequences introduce significant GPU memory overhead due to increased key-value (KV) cache. Existing KV cache compression methods mitigate memory bottlenecks but struggle in long reasoning tasks. In this paper, we analyze attention patterns in reasoning tasks and reveal a Token Importance Recurrence phenomenon: a large proportion of tokens regain high attention after multiple decoding steps, which is failed to capture by existing works and may lead to unpredictable eviction on such periodically critical tokens. To address this, we propose LazyEviction, an observation window-based lagged eviction framework retaining latent recurring tokens by prioritized eviction based on tokens' recurrence patterns. Extensive experiments demonstrate that LazyEviction reduces KV cache by 50%~70% while maintaining comparable accuracy, outperforming existing KV cache compression baselines. Our implementation code can be found at https://github.com/Halo-949/LazyEviction.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 37 of its 44 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

37 of 44 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning20 May
  2. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models9 Mar
  3. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  4. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  5. 2024Token-Budget-Aware LLM Reasoning24 Dec
  6. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  7. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct

Other references · 30

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  12. 2024Qwen2 Technical Reportcited by 2,120
18 more
  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  2. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  3. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  4. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  5. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  6. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  7. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  8. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  9. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  10. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  11. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inferencecited by 126
  12. 2024Transformers are Multi-State RNNscited by 120
  13. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 117
  14. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 116
  15. 2023FlashDecoding++: Faster Large Language Model Inference on GPUscited by 71
  16. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Timecited by 43
  17. 2025Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMscited by 20
  18. 2025TreeKV: Smooth Key-Value Cache Compression with Tree Structurescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 14.

16 papers.

Sort:

By area:

  1. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Modelscited by 21
  2. 2026TriAttention: Efficient Long Reasoning with Trigonometric KV Compressioncited by 21
  3. 2026MEMENTO: Teaching LLMs to Manage Their Own Contextcited by 18
  4. 2026SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoningcited by 11
  5. 2025Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Modelscited by 5
  6. 2026ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Modelscited by 3
  7. 2026HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compressioncited by 2
  8. 2026LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decodingcited by 1
  9. 2026Random Attention: Rethinking KV Cache Eviction for Efficient Reasoningcited by 1
  10. 2026State commitment learning: training language models to distinguish computation from memorycited by 0
  11. 2026GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Servingcited by 0
  12. 2026BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inferencecited by 0
4 more
  1. 2026StepKV: Step-Aware KV Cache Compression for LLM Agentscited by 0
  2. 2026CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cachecited by 0
  3. 2026AvoKV-E: Payload-Aware KV Cache Eviction for Long Reasoningcited by 0
  4. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025G-KV: Decoding-Time KV Cache Eviction with Global Attention
  2. 2026ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution
  3. 2025SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models
  4. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
  5. 2026Value-Aware Stochastic KV Cache Eviction for Reasoning Models
  6. 2025REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression
  7. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
  8. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning