mlsanityData to 7 Oct 2026

Unifying Masked Diffusion Models with Various Generation Orders and Beyond

arXiv 2602.02112 ↗First posted 2 Feb 2026cs.LG cs.AI cs.CLcited by 6

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked diffusion models (MDMs) are a potential alternative to autoregressive models (ARMs) for language generation, but generation quality depends critically on the generation order. Prior work either hard-codes an ordering (e.g., blockwise left-to-right) or learns an ordering policy for a pretrained MDM, which incurs extra cost and can yield suboptimal solutions due to the two-stage optimization. Motivated by this, we propose order-expressive masked diffusion model (OeMDM) for a broad class of diffusion generative processes with various generation orders, enabling the interpretation of MDM, ARM, and block diffusion in a single framework. Furthermore, building on OeMDM, we introduce learnable-order masked diffusion model (LoMDM), which jointly learns the generation ordering and diffusion backbone through a single objective from scratch, enabling the diffusion model to generate text in context-dependent ordering. Empirically, we confirm that LoMDM outperforms various discrete diffusion models across multiple language modeling benchmarks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

34 of 41 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing24 Oct
  2. 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies7 Oct
  3. 2025Dream 7B: Diffusion Large Language Models21 Aug
  4. 2025The Diffusion Duality12 Jun
  5. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  6. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  7. 2025Large Language Diffusion Models14 Feb
  8. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  9. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  10. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  11. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  12. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
2 more
  1. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  2. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 20

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2021Variational Diffusion Modelscited by 1,145
  10. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  11. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  12. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
8 more
  1. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  2. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  3. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  4. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  5. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  6. 2023Diffusion Models With Learned Adaptive Noisecited by 50
  7. 2025Learning-Order Autoregressive Models with Application to Molecular Graph Generationcited by 29
  8. 2025Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modelingcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokenscited by 2
  2. 2026The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Modelscited by 1
  3. 2026Understanding and Accelerating the Training of Masked Diffusion Language Modelscited by 0
  4. 2026Looped Diffusion Language Modelscited by 0
  5. 2026Length-Adaptive Decoding for Masked Diffusion Machine Translationcited by 0
  6. 2026Unmask the State: When Does State Adaptation Matter for Masked Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond
  2. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  3. 2025Theoretical Benefit and Limitation of Diffusion Language Model
  4. 2024Scaling up Masked Diffusion Models on Text
  5. 2026Auto-Regressive Masked Diffusion Models
  6. 2026Infinite Mask Diffusion for Few-Step Distillation
  7. 2026Multi-Mask Diffusion Language Models for Few-Step Generation
  8. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Models