mlsanityData to 14 Oct 2026

Zipage: Maintain High Request Concurrency for LLM Reasoning through Compressed PagedAttention

arXiv 2603.08743 ↗First posted 1 Mar 2026cs.DC cs.AIcited by 4

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

With reasoning becoming the generative paradigm for large language models (LLMs), the memory bottleneck caused by KV cache during the decoding phase has become a critical factor limiting high-concurrency service. Although existing KV cache eviction methods address the memory issue, most of them are impractical for industrial-grade applications. This paper introduces Compressed PagedAttention, a method that combines token-wise KV cache eviction with PagedAttention. We propose a comprehensive scheduling strategy and support prefix caching and asynchronous compression for Compressed PagedAttention. Based on this, we have developed a high-concurrency LLM inference engine, Zipage. On large-scale mathematical reasoning tasks, Zipage achieves around 95% of the performance of Full KV inference engines while delivering over 2.1× speedup.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 16 of its 27 references that are in the map. First posted on arXiv 1 Mar 2026.

Builds on

16 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025G-KV: Decoding-Time KV Cache Eviction with Global Attention29 Nov

Other references · 15

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  6. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  7. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  8. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  9. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  10. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 193
  11. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  12. 2025Shared Global and Local Geometry of Language Model Embeddingscited by 36
3 more
  1. 2024KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Headcited by 13
  2. 2024Core Context Aware Transformers for Long Context Language Modelingcited by 11
  3. 2025PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inferencecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026vToken: Token-Level Virtualization for Reclaimable KV Cachescited by 1
  2. 2026Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformerscited by 0
  3. 2026GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Servingcited by 0
  4. 2026Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
  2. 2026Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
  3. 2026ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoning
  4. 2026RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
  5. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
  6. 2025HAMburger: Accelerating LLM Inference via Token Smashing
  7. 2026Not All Thoughts Need HBM: Semantics-Aware Memory Hierarchy for LLM Reasoning
  8. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning