mlsanityData to 14 Oct 2026

Why mask diffusion does not work

arXiv 2510.03289 ↗First posted 29 Sep 2025cs.LG cs.AI cs.CLcited by 2

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

The main advantages of diffusion language models over autoregressive (AR) models lie in their ability to support parallel generation and bidirectional attention, enabling a more controllable generation process. In recent years, open-source mask diffusion language models have emerged, most of which are based on a variant known as absorbing diffusion. However, this paper demonstrates why mask diffusion faces inherent difficulties in achieving parallel generation and bidirectional attention. We also propose the most effective training and inference strategies for mask diffusion.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 10 of its 14 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

10 of 14 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  4. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  5. 2022Continuous diffusion for categorical data28 Nov
  6. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 4

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  3. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  4. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 148

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 13
  2. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024Simple and Effective Masked Diffusion Language Models
  2. 2025Diffusion Beats Autoregressive in Data-Constrained Settings
  3. 2025Attention Sinks in Diffusion Language Models
  4. 2026PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention
  5. 2025Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models
  6. 2026Scaling Beyond Masked Diffusion Language Models
  7. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  8. 2026Auto-Regressive Masked Diffusion Models