mlsanityData to 14 Oct 2026

Variational Masked Diffusion Models

arXiv 2510.23606 ↗First posted 27 Oct 2025cs.LG cs.AI cs.CLcited by 4

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked diffusion models have recently emerged as a flexible framework for discrete generative modeling. However, a key limitation of standard masked diffusion is its inability to effectively capture dependencies among tokens that are predicted concurrently, leading to degraded generation quality when dependencies among tokens are important. To explicitly model dependencies among tokens, we propose Variational Masked Diffusion (VMD), a framework that introduces latent variables into the masked diffusion process. Through controlled experiments on synthetic datasets, we demonstrate that VMD successfully learns dependencies that conventional masked diffusion fails to capture. We further validate the effectiveness of our approach on Sudoku puzzles and text datasets, where learning of dependencies among tokens improves global consistency. Across these domains, VMD enhances both generation quality and dependency awareness, highlighting the value of integrating variational inference into masked diffusion. Our code is available at: https://riccizz.github.io/VMD.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 35 of its 39 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

35 of 39 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  2. 2025Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithms10 Mar
  3. 2025Large Language Diffusion Models14 Feb
  4. 2025Theoretical Benefit and Limitation of Diffusion Language Model13 Feb
  5. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  6. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  7. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
  8. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  9. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  11. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  12. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
3 more
  1. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  2. 2022Continuous diffusion for categorical data28 Nov
  3. 2021Autoregressive Diffusion Models5 Oct

Other references · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  6. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  7. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  8. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  10. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
  11. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  12. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
8 more
  1. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  2. 2024Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzlescited by 38
  3. 2025Variational Rectified Flow Matchingcited by 35
  4. 2025Towards Hierarchical Rectified Flowcited by 20
  5. 2025DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generationcited by 10
  6. 2025Hierarchical Rectified Flow Matching with Mini-Batch Couplingscited by 7
  7. 2024Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelinescited by 6
  8. 2025Plug-and-Play Context Feature Reuse for Efficient Masked Generationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026CORE: Context-Robust Remasking for Diffusion Language Modelscited by 10
  2. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculationcited by 5
  3. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 4
  4. 2026Few-Step Diffusion Language Models via Trajectory Self-Distillationcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  2. 2026Learning Generation Orders for Masked Discrete Diffusion Models via Variational Inference
  3. 2026Latent Shadows: The Gaussian-Discrete Duality in Masked Diffusion
  4. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction
  5. 2026VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editing
  6. 2026Unifying Masked Diffusion Models with Various Generation Orders and Beyond
  7. 2026On the Trainability of Masked Diffusion Language Models via Blockwise Locality
  8. 2025Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generator