mlsanityData to 14 Oct 2026

Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs

arXiv 2605.18165 ↗First posted 18 May 2026cs.LGcited by 1

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Unlike autoregressive models, which generate one token at a time, dLLMs denoise a chunk of [MASK] tokens jointly and sample one or more tokens per step; despite enabling parallel decoding, this process incurs substantial computational cost due to the large chunk size of masked tokens. We observe that much of this cost is spent on repeatedly processing the preceding context and many [MASK] tokens with the same feature representations, indicating considerable computational redundancy. In this work, we revisit dLLM's redundancy from the perspective of [MASK] tokens. Through systematic analysis, we verify the redundancy of [MASK] tokens while revealing their critical role in providing structural information. Guided by these findings, we propose position-preserving [MASK] token compression and terminal-aware augmentation. By compressing redundant [MASK] computation, this approach accelerates decoding and further provides a natural extension toward context-folding-like long-context scaling under limited input-length constraints for full-sequence dLLMs such as LLaDA-8B-Instruct and LLaDA-1.5. Moreover, for block dLLMs such as LLaDA2.0-mini, it augments the context with a protected terminal [MASK] token to enhance generation quality with negligible overhead.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 36 of its 37 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

36 of 37 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Attention Is All You Need for KV Cache in Diffusion LLMs16 Oct
  2. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching27 Sep
  3. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
  4. 2025dKV-Cache: The Cache for Diffusion Language Models21 May

Other references · 32

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  6. 2025Large Language Diffusion Modelscited by 912
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  8. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  10. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  11. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  12. 2025Dream 7B: Diffusion Large Language Modelscited by 544
20 more
  1. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  2. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  3. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  4. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  5. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  6. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  7. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  8. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  9. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
  10. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 148
  11. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 144
  12. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
  13. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
  14. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusioncited by 77
  15. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 39
  16. 2025QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generationcited by 21
  17. 2025SparseD: Sparse Attention for Diffusion Language Modelscited by 17
  18. 2025Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Modelscited by 12
  19. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skippingcited by 6
  20. 2026FlashBlock: Attention Caching for Efficient Long-Context Block Diffusioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Affix Cache for Diffusion Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models
  2. 2025A Survey of Token Compression for Efficient Multimodal Large Language Models
  3. 2026MDLMPE: Distribution Aware Positional Encoding for Masked Diffusion Language Models
  4. 2025Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference
  5. 2026Reconsidering Positional Supervision in Masked Diffusion Language Model Training
  6. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
  7. 2026x-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models
  8. 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching