mlsanityData to 14 Oct 2026

Minima-KV: Retention-Preserving KV Cache Compression with Mixed-Format Paged Attention

arXiv 2608.23834 ↗First posted 24 Aug 2026cs.AIcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

The key-value (KV) cache is a primary capacity and bandwidth bottleneck in long-context LLM serving. We present Minima-KV, a retention-preserving hierarchy for mixed-format paged attention. Recent and protected Anchor pages remain in FP8, while older non-anchor pages move to packed TQ3; every live-request page remains addressable. Format-specific kernels compute partial attention states and combine them through a globally normalized online-softmax merge, enabling direct heterogeneous decode without a cache-sized dense shadow. Across separate, configuration-bound Qwen3.6-27B profiles on a single 96-GB NVIDIA RTX PRO 6000 Blackwell GPU, deployment accounting reports 18.3 KiB of attention KV per live token, corresponding to 3.50x compression relative to BF16 and 1.75x relative to FP8. A materializing quality profile matches its dense control on 16K RULER needle-in-a-haystack tasks. On the same 503-question LongBench v2 set, measured deltas are -0.80, -0.60, and -0.40 percentage points at 16K, 32K, and 64K. A separate single-pair direct-decode canary with two 59,008-token requests measures 3.625x active-KV compression and 0.9821x throughput relative to its control, routes all 16 full-attention layers without fallback, and retains no dense shadow. These results establish a practical mixed-format path for compressing long-context state without evicting live-request KV pages.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 24 of its 36 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

24 of 36 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression1 Jul
  2. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference8 Oct
  3. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs19 Dec
  4. 2024DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction4 Dec
  5. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference28 Oct
  6. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
  7. 2024Palu: Compressing KV-Cache with Low-Rank Projection30 Jul
  8. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  10. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM8 Mar
  11. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization28 Feb
  12. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
2 more
  1. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  2. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 10

  1. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  2. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  3. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  4. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  5. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 196
  6. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
  7. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 93
  8. 2025TileLang: A Composable Tiled Programming Model for AI Systemscited by 43
  9. 2026RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cachecited by 4
  10. 2026QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decodingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
  2. 2026Tangram: Unlocking Non-Uniform KV Cache Compression for Efficient Multi-turn LLM Serving
  3. 2026Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
  4. 2026ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression
  5. 2025FlexiCache: Leveraging Temporal Stability of Attention Heads for Efficient KV Cache Management
  6. 2026PersistentKV: Page-Aware Decode Scheduling for Long-Context LLM Serving on Commodity GPUs
  7. 2026CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
  8. 2026More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression