mlsanityData to 7 Oct 2026

Diffusion Language Models Generation Can Be Halted Early

arXiv 2305.10818 ↗First posted 18 May 2023cs.LG cs.CLcited by 3

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language models (DLMs) are a promising avenue for text generation due to their practical properties on tractable controllable generation. They also have the advantage of not having to predict text autoregressively. However, despite these notable features, DLMs have not yet reached the performance levels of their autoregressive counterparts. One of the ways to reduce the performance gap between these two types of language models is to speed up the generation of DLMs. Therefore, we propose a novel methodology to address this issue in this work. It enables the execution of more generation steps within a given time frame, leading to higher-quality outputs. Specifically, our methods estimate DLMs completeness of text generation and allow adaptive halting of the generation process. We evaluate our methods on Plaid, SSD, and CDCD DLMs and create a cohesive perspective on their generation workflows. Finally, we confirm that our methods allow halting these models and decrease the generation time by 10-40% without a drop in the quality of model samples.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 39 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

27 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  2. 2022Self-conditioned Embedding Diffusion for Text Generation8 Nov
  3. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  11. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  12. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
11 more
  1. 2021Variational Diffusion Modelscited by 1,145
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  3. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  4. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  5. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  6. 2017Non-Autoregressive Neural Machine Translationcited by 457
  7. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  8. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  9. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 224
  10. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  11. 2022PALBERT: Teaching ALBERT to Pondercited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Differences in Text Generated by Diffusion and Autoregressive Language Modelscited by 1
  2. 2026ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Taskscited by 1
  3. 2026Reducing Pretraining-Generation Mismatch in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Diffusion Language Models
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  3. 2024Energy-Based Diffusion Language Models for Text Generation
  4. 2026Diffusion Language Models: An Experimental Analysis
  5. 2025Theoretical Benefit and Limitation of Diffusion Language Model
  6. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
  7. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
  8. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models