Masked Diffusion Models as Energy Minimization
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
We present a systematic theoretical framework that interprets masked diffusion models (MDMs) as solutions to energy minimization problems in discrete optimal transport. Specifically, we prove that three distinct energy formulations--kinetic, conditional kinetic, and geodesic energy--are mathematically equivalent under the structure of MDMs, and that MDMs minimize all three when the mask schedule satisfies a closed-form optimality condition. This unification not only clarifies the theoretical foundations of MDMs, but also motivates practical improvements in sampling. By parameterizing interpolation schedules via Beta distributions, we reduce the schedule design space to a tractable 2D search, enabling efficient post-training tuning without model modification. Experiments on synthetic and real-world benchmarks demonstrate that our energy-inspired schedules outperform hand-crafted baselines, particularly in low-step sampling settings.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 39 of its 48 references that are in the map. First posted on arXiv 17 Sep 2025.
Builds on
39 of 48 references.
Closest ancestors · 9
Same direction, newest first
- 2025Effective and Efficient Masked Image Generation Models10 Mar
- 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms1 Feb
- 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Models10 Oct
- 2024Informed Correctors for Discrete Diffusion Models30 Jul
- 2023Formulating Discrete Probability Flow Through Optimal Transport7 Nov
- 2022Score-based Continuous-time Discrete Diffusion Models30 Nov
- 2022A Continuous Time Framework for Discrete Denoising Models30 May
- 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
- 2021Variational Diffusion Models1 Jul
Other references · 30
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
- 2022MaskGIT: Masked Generative Image Transformercited by 1,099
18 more
- 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
- 2025Large Language Diffusion Modelscited by 912
- 2024Simple and Effective Masked Diffusion Language Modelscited by 848
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
- 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
- 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
- 2024Qwen2-Audio Technical Reportcited by 560
- 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
- 2024Discrete Flow Matchingcited by 317
- 2022Continuous diffusion for categorical datacited by 214
- 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
- 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 148
- 2024Diffusion Language Models Are Versatile Protein Learnerscited by 98
- 2024CLLMs: Consistency Large Language Modelscited by 72
- 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 64
- 2024Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspectivecited by 57
- 2023On Kinetic Optimal Probability Paths for Generative Modelscited by 42
- 2025Continuous Diffusion Model for Language Modelingcited by 31
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025On scalable and efficient training of diffusion samplers
- 2026Schedule optimization for tau-leaping in masked discrete diffusion
- 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations
- 2025Path Planning for Masked Diffusion Model Sampling
- 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion
- 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Models
- 2025DiffPro: Joint Timestep and Layer-Wise Precision Optimization for Efficient Diffusion Inference
- 2026Efficient Diffusion Models under Nonconvex Equality and Inequality constraints via Landing