# Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference

arXiv: https://arxiv.org/abs/2510.09309 · HTML: https://mlsanity.com/paper/2510.09309/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving.
- First posted on arXiv 10 Oct 2025.

## Placed in

- Efficient LLM inference and serving / [Diffusion LLM and MoE inference](https://mlsanity.com/inference/diffusion-llm-and-moe-inference/)

## Built on by (most cited first, 17 of 17)

- [DMax: Aggressive Parallel Decoding for dLLMs](https://mlsanity.com/paper/2604.08302/) (2026)
- [Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing](https://mlsanity.com/paper/2602.02159/) (2026)
- [Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM Serving](https://mlsanity.com/paper/2512.17077/) (2025)
- [Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation](https://mlsanity.com/paper/2607.14557/) (2026)
- [Unlocking the Potential of Diffusion Language Models through Template Infilling](https://mlsanity.com/paper/2510.13870/) (2025)
- [Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models](https://mlsanity.com/paper/2604.08964/) (2026)
- [Reasoning in Diffusion Large Language Models is Concentrated in Dynamic Confusion Zones](https://mlsanity.com/paper/2511.15208/) (2025)
- [Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models](https://mlsanity.com/paper/2602.10953/) (2026)
- [MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM](https://mlsanity.com/paper/2602.14209/) (2026)
- [FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration](https://mlsanity.com/paper/2605.20022/) (2026)
- [GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference](https://mlsanity.com/paper/2609.17573/) (2026)
- [STDec: Spatio-Temporal Stability Guided Decoding for dLLMs](https://mlsanity.com/paper/2604.06330/) (2026)
- [Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques](https://mlsanity.com/paper/2607.12829/) (2026)
- [Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs](https://mlsanity.com/paper/2608.22367/) (2026)
- [Affix Cache for Diffusion Large Language Models](https://mlsanity.com/paper/2608.26140/) (2026)
- [Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs](https://mlsanity.com/paper/2609.26796/) (2026)
- [Mask-Guided KV Cache Eviction in Block Diffusion Language Models](https://mlsanity.com/paper/2610.06996/) (2026)

## Abstract (arXiv)

Diffusion large language models (dLLMs) present a promising alternative to dominant autoregressive models (ARMs) by the ability of parallel decoding at the expense of substantial computation and memory costs. Specifically, the cache mechanism for bidirectional attention in dLLMs demands large memory footprint, restricting their ability to handle long contexts under resource-limited settings. Existing cache eviction strategies are designed for ARMs and ignore the unique characteristics of dLLMs, thus leading to unsatisfactory performance. To address these challenges, we introduce MaskKV, a training-free cache eviction framework tailored to dLLMs, focusing on the effect of mask tokens in dLLMs. MaskKV is built on two key innovations: (1) a mask-query guided scoring mechanism that leverages attention weights to identify and evict less critical prompt tokens for each head; (2) an adaptive cache budgeting strategy that improves efficiency by reducing allocation in intermediate layers and concentrating resources on prompt-preferring heads. On LLaDA with MaskKV, compressing the KV cache to only 256 pairs (less than 5% of tokens) retains 94% of the full-cache performance on LongBench and achieves up to 31x acceleration at 32k prompt length. The code is publicly available at: https://github.com/jianuo-huang/MaskKV