mlsanityData to 14 Oct 2026

Understanding the Quality-Diversity Trade-off in Diffusion Language Models

arXiv 2503.10683 ↗First posted 11 Mar 2025cs.CL cs.AI cs.LGcited by 1

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion models have seen immense success in modelling continuous data across a range of domains such as vision and audio. Despite the challenges of adapting diffusion models to discrete data, recent work explores their application to text generation by working in the continuous embedding space. However, these models lack a natural means to control the inherent trade-off between quality and diversity as afforded by the temperature hyperparameter in autoregressive models, hindering understanding of model performance and restricting generation quality. This work proposes the use of classifier-free guidance and stochastic clamping for manipulating the quality-diversity trade-off on sequence-to-sequence tasks, demonstrating that these techniques may be used to improve the performance of a diffusion language model.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 21 of its 26 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

21 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  2. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  3. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  8. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  9. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  10. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  11. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  12. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
6 more
  1. 2017Why We Need New Evaluation Metrics for NLGcited by 233
  2. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
  3. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  4. 2023Elucidating the Exposure Bias in Diffusion Modelscited by 101
  5. 2023Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Stepscited by 73
  6. 2023DiffuSum: Generation Enhanced Extractive Summarization with Diffusioncited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Recovering molecules from coarse-grained beads: free-energy-conditioned generative backmapping across chemical spacecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  2. 2026Diffusion Language Models: An Experimental Analysis
  3. 2023Diffusion Language Models Generation Can Be Halted Early
  4. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control
  5. 2026Free Lunch for Pass@k? Low Cost Diverse Sampling for Diffusion Language Models
  6. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
  7. 2025Discrete Diffusion Models for Language Generation
  8. 2022Self-conditioned Embedding Diffusion for Text Generation