mlsanityData to 7 Oct 2026

Balancing Understanding and Generation in Discrete Diffusion Models

arXiv 2602.01362 ↗First posted 1 Feb 2026cs.CLcited by 3

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

In discrete generative modeling, two dominant paradigms demonstrate divergent capabilities: Masked Diffusion Language Models (MDLM) excel at semantic understanding and zero-shot generalization, whereas Uniform-noise Diffusion Language Models (UDLM) achieve strong few-step generation quality, yet neither attains balanced performance across both dimensions. To address this, we propose XDLM, which bridges the two paradigms via a stationary noise kernel. XDLM offers two key contributions: (1) it provides a principled theoretical unification of MDLM and UDLM, recovering each paradigm as a special case; and (2) an alleviated memory bottleneck enabled by an algebraic simplification of the posterior probabilities. Experiments demonstrate that XDLM advances the Pareto frontier between understanding capability and generation quality. Quantitatively, XDLM surpasses UDLM by 5.4 points on zero-shot text benchmarks and outperforms MDLM in few-step image generation (FID 54.1 vs. 80.8). When scaled to tune an 8B-parameter large language model, XDLM achieves 15.0 MBPP in just 32 steps, effectively doubling the baseline performance. Finally, analysis of training dynamics reveals XDLM's superior potential for long-term scaling. Code is available at https://github.com/MzeroMiko/XDLM

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 29 of its 37 references that are in the map. First posted on arXiv 1 Feb 2026.

Builds on

29 of 37 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Dream 7B: Diffusion Large Language Models21 Aug
  2. 2025The Diffusion Duality12 Jun
  3. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  4. 2025Generalized Interpolating Discrete Diffusion6 Mar
  5. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  6. 2025Large Language Diffusion Models14 Feb
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2023Likelihood-Based Diffusion Language Models30 May

Other references · 20

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  10. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  11. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  12. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
8 more
  1. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  2. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  3. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  4. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  5. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  6. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  7. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  8. 2025Unifying Autoregressive and Diffusion-Based Sequence Generationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Provably adaptive sampling with uniform and remasking discrete diffusion modelscited by 3
  2. 2026DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMscited by 1
  3. 2026Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
  2. 2025Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model
  3. 2026Unifying Masked Diffusion Models with Various Generation Orders and Beyond
  4. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
  5. 2026Scaling Beyond Masked Diffusion Language Models
  6. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
  7. 2026Bridging the Discrete-Continuous Gap: Unified Multimodal Generation via Coupled Manifold Discrete Absorbing Diffusion
  8. 2026Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design