mlsanityData to 7 Oct 2026

GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

arXiv 2609.03494 ↗First posted 3 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Long-output reasoning has made the key--value (KV) cache a critical memory bottleneck for efficient LLM serving. Existing KV compression methods usually rely on a predefined per-request budget and adjust only which KV states are retained, leaving the total capacity fixed throughout decoding. However, reasoning workloads exhibit substantial demand variation: different requests require different KV capacities, and the attention demand of an individual request evolves during generation. We introduce GrowPage, an on-demand KV budgeting framework that treats KV capacity as a runtime resource. GrowPage maintains lightweight dual-timescale query summaries to capture recent and long-term attention behaviors, and uses their relative attention working sets to estimate demand evolution. At each capacity boundary, GrowPage either compresses KV states within the current allocation or acquires an additional physical page when broader demand emerges. By integrating with PagedAttention's page-level memory abstraction, GrowPage preserves continuous batching and prefix caching. Experiments on reasoning benchmarks across multiple models show that GrowPage achieves a superior performance--throughput trade-off over existing approaches.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 40 of its 44 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

40 of 44 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026Information-Aware KV Cache Compression for Long Reasoning25 Jun
  2. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression1 May
  3. 2026LongFlow: Efficient KV Cache Compression for Reasoning Models12 Mar
  4. 2026Zipage: Maintain High Request Concurrency for LLM Reasoning through Compressed PagedAttention1 Mar
  5. 2025G-KV: Decoding-Time KV Cache Eviction with Global Attention29 Nov
  6. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models1 Oct
  7. 2025REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression14 Aug
  8. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning19 Jun
  9. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Models30 May
  10. 2025RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning16 Feb

Other references · 30

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  10. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  11. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  12. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
18 more
  1. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 466
  2. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  3. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  4. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 391
  5. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 283
  6. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 242
  7. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  8. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  9. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  10. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
  11. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inferencecited by 124
  12. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Modelscited by 122
  13. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantizationcited by 99
  14. 2024vAttention: Dynamic Memory Management for Serving LLMs without PagedAttentioncited by 93
  15. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Poolcited by 67
  16. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Timecited by 43
  17. 2025Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMscited by 20
  18. 2025PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inferencecited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoning
  2. 2025Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving
  3. 2026OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching
  4. 2026StepKV: Step-Aware KV Cache Compression for LLM Agents
  5. 2026ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Models
  6. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  7. 2026Adaptive Mass-Segmented KV Compression for Long-Context Reasoning
  8. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning