mlsanityData to 14 Oct 2026

DLM-One: Diffusion Language Models for One-Step Sequence Generation

arXiv 2506.00290 ↗First posted 30 May 2025cs.CL cs.LG stat.MLcited by 20

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

This paper introduces DLM-One, a score-distillation-based framework for one-step sequence generation with continuous diffusion language models (DLMs). DLM-One eliminates iterative refinement by aligning the scores of a student model's outputs with the score function of a pretrained teacher DLM in the forward-diffused noisy space. We demonstrate that our framework is architecture-agnostic and robust across diverse continuous manifolds, including standard token embedding spaces and logit simplex spaces. Through experiments on multiple representative DLMs, we show that DLM-One achieves up to ~2000× speedup in sampling steps and ~500× in wall-clock time, while maintaining competitive performance on benchmark text generation tasks. We further analyze failure modes in language-domain diffusion distillation and propose an adversarially-regularized two-stage training scheme to prevent student degeneration. Our findings position one-step score distillation as a viable path for the efficient deployment of continuous diffusion models operating in continuous space for natural language processing.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 81 of its 96 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

81 of 96 references.

Closest ancestors · 29

Same direction, newest first

  1. 2025TESS 2: A Large-Scale Generalist Diffusion Language Model19 Feb
  2. 2025Large Language Diffusion Models14 Feb
  3. 2024Scaling up Masked Diffusion Models on Text24 Oct
  4. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  5. 2024Discrete Copula Diffusion2 Oct
  6. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  10. 2024TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings29 Feb
  11. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  12. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model5 Jun
17 more
  1. 2023Likelihood-Based Diffusion Language Models30 May
  2. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation16 May
  3. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusion15 May
  4. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation6 May
  5. 2023Diffusion Models for Non-autoregressive Text Generation: A Survey12 Mar
  6. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises20 Feb
  7. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  8. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise22 Dec
  9. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  10. 2022Latent Diffusion for Language Generation19 Dec
  11. 2022Empowering Diffusion Models on the Embedding Space for Text Generation19 Dec
  12. 2022Continuous diffusion for categorical data28 Nov
  13. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  14. 2022Self-conditioned Embedding Diffusion for Text Generation8 Nov
  15. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct
  16. 2022DiffusER: Discrete Diffusion via Edit-based Reconstruction30 Oct
  17. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2024The Llama 3 Herd of Modelscited by 14,715
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2020Denoising Diffusion Implicit Modelscited by 9,781
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  6. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  8. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  9. 2023Consistency Modelscited by 1,822
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  12. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  13. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  14. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  15. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  16. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  17. 2023Adversarial Diffusion Distillationcited by 759
  18. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  19. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
  20. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  21. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  22. 2023Improved Techniques for Training Consistency Modelscited by 439
  23. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  24. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 273
  25. 2023Challenges and Applications of Large Language Modelscited by 254
  26. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 236
  27. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 214
  28. 2022Diffusion-GAN: Training GANs with Diffusioncited by 205
  29. 2024The pitfalls of next-token predictioncited by 160
  30. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 148
  31. 2024Consistency Models Made Easycited by 144
  32. 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
  33. 2024Multistep Distillation of Diffusion Models via Moment Matchingcited by 82
  34. 2020Limitations of Autoregressive Models and Their Alternativescited by 55
  35. 2017Quasar: Datasets for Question Answering by Search and Readingcited by 50
  36. 2024Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Stepcited by 47
  37. 2021Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generationcited by 40
  38. 2025Few-Step Diffusion via Score identity Distillationcited by 12

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 15.

20 papers.

Sort:

By area:

  1. 2025A Survey on Diffusion Language Modelscited by 78
  2. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Modelscited by 17
  3. 2026Continuous Latent Diffusion Language Modelcited by 14
  4. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Modelscited by 13
  5. 2025Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedulescited by 11
  6. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practicescited by 10
  7. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Modelscited by 8
  8. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decodingcited by 7
  9. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 7
  10. 2026Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Modelscited by 3
  11. 2026FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Versioncited by 2
  12. 2026Coupling Models for One-Step Discrete Generationcited by 2
8 more
  1. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or Onecited by 2
  2. 2026GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Modelscited by 1
  3. 2026VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Terminationcited by 1
  4. 2026FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptationcited by 0
  5. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Modelscited by 0
  6. 2026SelFusion: Self-distillation for Diffusion Language Modelscited by 0
  7. 2026Distribution Matching Distillation for Continuous Diffusion Language Modelscited by 0
  8. 2026Representation-Space MMD for Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Language Models: An Experimental Analysis
  2. 2026Hierarchical Continuous Diffusion Language Models
  3. 2023Diffusion Language Models Generation Can Be Halted Early
  4. 2026DLM-SWAI: Steering Diffusion Language Models Before They Unmask
  5. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
  6. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  7. 2025Theoretical Benefit and Limitation of Diffusion Language Model
  8. 2026IDLM: Inverse-distilled Diffusion Language Models