mlsanityData to 14 Oct 2026

ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Models

arXiv 2606.11164 ↗First posted 9 Jun 2026cs.AIcited by 3

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Long chain-of-thought (CoT) trajectories in large language model (LLM) reasoning cause severe inference bottlenecks due to rapid key-value (KV) cache growth. Current decoding-time compression methods mitigate this issue via token eviction, but typically assume a uniform budget distribution across all layers and heads. In contrast, existing non-uniform budget allocation methods are predominantly designed for the static prompt prefill phase, and they do not capture the stepwise context demands of autoregressive reasoning. To bridge this gap, we propose ReasonAlloc, a training-free framework that recasts decoding-time KV compression as a hierarchical budget allocation problem. ReasonAlloc operates at two complementary levels: an offline layer-wise preallocation strategy captures an architecture-driven demand pattern which we call ``textitReasoning Wave'', while an online head-wise strategy reallocates resources during decoding to information-rich heads based on real-time utility. Evaluations on mathematical reasoning benchmarks (MATH-500, AIME~2024) using DeepSeek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Qwen-14B, and AceReason-14B show that ReasonAlloc outperforms uniform-budget R-KV, SnapKV, and Pyramid-RKV (a baseline enforcing a static, monotonically decreasing layer budget), with the largest gains at small budgets (128-512 tokens). ReasonAlloc is plug-and-play with existing token-eviction policies and introduces negligible inference-time overhead.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 19 of its 22 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

19 of 22 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning19 Jun
  2. 2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models6 May
  3. 2025Concise Reasoning via Reinforcement Learning7 Apr
  4. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning2 Apr

Other references · 15

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  6. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  7. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  9. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  10. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  11. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Modelscited by 125
  12. 2023Contrastive Chain-of-Thought Promptingcited by 52
3 more
  1. 2023Extending Context Window of Large Language Models via Semantic Compressioncited by 45
  2. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMscited by 42
  3. 2025Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inferencecited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decodingcited by 1
  2. 2026WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Mindscited by 0
  3. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  2. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
  3. 2026GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving
  4. 2026CASK: Core-Aware Selective KV Compression for Reasoning Traces
  5. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
  6. 2026StepKV: Step-Aware KV Cache Compression for LLM Agents
  7. 2026SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning
  8. 2025Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving