mlsanityData to 7 Oct 2026

Adaptive Soft Rolling KV Freeze with Entropy-Guided Recovery: Sublinear Memory Growth for Efficient LLM Inference

arXiv 2512.11221 ↗First posted 12 Dec 2025cs.LG cs.AI cs.CLcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

We present Adaptive Soft Rolling KV Freeze with Entropy-Guided Recovery (ASR-KF-EGR), a training-free inference-time framework for efficient large language model generation. Our method introduces a reversible soft-freeze mechanism that temporarily suspends key-value (KV) updates for low-importance tokens identified within a sliding attention window. Unlike eviction-based approaches that permanently discard context, ASR-KF-EGR preserves all tokens in off-GPU storage and restores them on demand. We extend the framework with sublinear freeze scheduling, where freeze duration grows sublinearly with repeated low-importance detections, preventing over-aggressive compression. Preliminary experiments on LLaMA-3 8B demonstrate 55-67% reduction in active KV cache size while maintaining generation quality and passing needle-in-haystack retrieval tests. The method is architecture-agnostic, requires no fine-tuning, and provides a practical solution for memory-constrained deployment of long-context LLMs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 8 of its 8 references that are in the map. First posted on arXiv 12 Dec 2025.

Builds on

8 of 8 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  2. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 6

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  6. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2026SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference
  2. 2025KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
  3. 2026Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
  4. 2025EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
  5. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
  6. 2025WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference
  7. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  8. 2025Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPU