mlsanityData to 7 Oct 2026

Masked Diffusion Models are Secretly Learned-Order Autoregressive Models

arXiv 2511.19152 ↗First posted 24 Nov 2025cs.LG stat.MLcited by 3

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked Diffusion Models (MDMs) have emerged as one of the most promising paradigms for generative modeling over discrete domains. It is known that MDMs effectively train to decode tokens in a random order, and that this ordering has significant performance implications in practice. This observation raises a fundamental question: can we design a training framework that optimizes for a favorable decoding order? We answer this in the affirmative, showing that the continuous-time variational objective of MDMs, when equipped with multivariate noise schedules, can identify and optimize for a decoding order during training. We establish a direct correspondence between decoding order and the multivariate noise schedule and show that this setting breaks invariance of the MDM objective to the noise schedule. Furthermore, we prove that the MDM objective decomposes precisely into a weighted auto-regressive losses over these orders, which establishes them as auto-regressive models with learnable orders.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 38 of its 40 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

38 of 40 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey16 Jun
  2. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  3. 2025Generalized Interpolating Discrete Diffusion6 Mar
  4. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  5. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  6. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  10. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  11. 2022Continuous diffusion for categorical data28 Nov
  12. 2021Autoregressive Diffusion Models5 Oct

Other references · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  8. 2021Variational Diffusion Modelscited by 1,145
  9. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  10. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  11. 2024Discrete Flow Matchingcited by 301
  12. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
14 more
  1. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  2. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  3. 2024TabDiff: a Mixed-type Diffusion Model for Tabular Data Generationcited by 60
  4. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  5. 2024Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspectivecited by 54
  6. 2025Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Modelcited by 52
  7. 2023Diffusion Models With Learned Adaptive Noisecited by 50
  8. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 33
  9. 2025Learning-Order Autoregressive Models with Application to Molecular Graph Generationcited by 29
  10. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusioncited by 14
  11. 2025Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generationcited by 9
  12. 2025Discrete Diffusion Trajectory Alignment via Stepwise Decompositioncited by 9
  13. 2023Let There Be Order: Rethinking Ordering in Autoregressive Graph Generationcited by 7
  14. 2022Permutation-Invariant Tabular Data Synthesiscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026DiBS: Diffusion-Informed Branch Selectioncited by 0
  2. 2026Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Modelscited by 0
  3. 2026UNIFUSION: Adapting Autoregressive Language Models into Discrete Diffusion under a Unified Reverse-Rate Objectivecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Bringing Stability to Diffusion: Decomposing and Reducing Variance of Training Masked Diffusion Models
  2. 2026Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective
  3. 2026Auto-Regressive Masked Diffusion Models
  4. 2025Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
  5. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  6. 2026Unifying Masked Diffusion Models with Various Generation Orders and Beyond
  7. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  8. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations