mlsanityData to 14 Oct 2026

Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference

arXiv 2510.09309 ↗First posted 10 Oct 2025cs.CLcited by 17

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion large language models (dLLMs) present a promising alternative to dominant autoregressive models (ARMs) by the ability of parallel decoding at the expense of substantial computation and memory costs. Specifically, the cache mechanism for bidirectional attention in dLLMs demands large memory footprint, restricting their ability to handle long contexts under resource-limited settings. Existing cache eviction strategies are designed for ARMs and ignore the unique characteristics of dLLMs, thus leading to unsatisfactory performance. To address these challenges, we introduce MaskKV, a training-free cache eviction framework tailored to dLLMs, focusing on the effect of mask tokens in dLLMs. MaskKV is built on two key innovations: (1) a mask-query guided scoring mechanism that leverages attention weights to identify and evict less critical prompt tokens for each head; (2) an adaptive cache budgeting strategy that improves efficiency by reducing allocation in intermediate layers and concentrating resources on prompt-preferring heads. On LLaDA with MaskKV, compressing the KV cache to only 256 pairs (less than 5% of tokens) retains 94% of the full-cache performance on LongBench and achieves up to 31x acceleration at 32k prompt length. The code is publicly available at: https://github.com/jianuo-huang/MaskKV

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. First posted on arXiv 10 Oct 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 14.

17 papers.

Sort:

By area:

  1. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  2. 2026Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusingcited by 7
  3. 2025Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM Servingcited by 5
  4. 2026Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncationcited by 5
  5. 2025Unlocking the Potential of Diffusion Language Models through Template Infillingcited by 4
  6. 2026Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Modelscited by 4
  7. 2025Reasoning in Diffusion Large Language Models is Concentrated in Dynamic Confusion Zonescited by 3
  8. 2026Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Modelscited by 3
  9. 2026MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLMcited by 2
  10. 2026FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibrationcited by 2
  11. 2026GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inferencecited by 2
  12. 2026STDec: Spatio-Temporal Stability Guided Decoding for dLLMscited by 1
5 more
  1. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  2. 2026Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMscited by 0
  3. 2026Affix Cache for Diffusion Large Language Modelscited by 0
  4. 2026Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMscited by 0
  5. 2026Mask-Guided KV Cache Eviction in Block Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
  2. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
  3. 2026Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
  4. 2025Fast-dLLM v2: Efficient Block-Diffusion LLM
  5. 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
  6. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
  7. 2025dKV-Cache: The Cache for Diffusion Language Models
  8. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping