mlsanityData to 7 Oct 2026

SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control

arXiv 2210.17432 ↗First posted 31 Oct 2022cs.CL cs.LGcited by 191

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Despite the growing success of diffusion models in continuous-valued domains (e.g., images), similar efforts for discrete domains such as text have yet to match the performance of autoregressive language models. In this work, we present SSD-LM -- a diffusion-based language model with two key design choices. First, SSD-LM is semi-autoregressive, iteratively generating blocks of text, allowing for flexible output length at decoding time while enabling local bidirectional context updates. Second, it is simplex-based, performing diffusion on the natural vocabulary space rather than a learned latent space, allowing us to incorporate classifier guidance and modular control using off-the-shelf classifiers without any adaptation. We evaluate SSD-LM on unconstrained text generation benchmarks, and show that it matches or outperforms strong autoregressive GPT-2 models across standard quality and diversity metrics, while vastly outperforming diffusion-based baselines. On controlled text generation, SSD-LM also outperforms competitive baselines, with an extra advantage in modularity.

Key facts. Cited by 191 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 73 of its 98 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

73 of 98 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
38 more
  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  2. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2022Video Diffusion Modelscited by 2,280
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  13. 2018Hierarchical Neural Story Generationcited by 1,182
  14. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  15. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  16. 2021Ethical and social risks of harm from Language Modelscited by 957
  17. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  18. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  19. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  20. 2022On Distillation of Guided Diffusion Modelscited by 721
  21. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  22. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  23. 2019Defending Against Neural Fake Newscited by 553
  24. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  25. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  26. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  27. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  28. 2017Non-Autoregressive Neural Machine Translationcited by 457
  29. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  30. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  31. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  32. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  33. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  34. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  35. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  36. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  37. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  38. 2021Societal Biases in Language Generation: Progress and Challengescited by 201

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 20; 2024: 32; 2025: 64; 2026: 73.

191 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  3. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  4. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  5. 2024Discrete Flow Matchingcited by 301
  6. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  7. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  8. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  9. 2022Continuous diffusion for categorical datacited by 202
  10. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 201
  11. 2024Scaling up Masked Diffusion Models on Textcited by 195
  12. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
38 more
  1. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  2. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  3. 2023Likelihood-Based Diffusion Language Modelscited by 147
  4. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  5. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 140
  6. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  7. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  8. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  9. 2025The Diffusion Dualitycited by 124
  10. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  11. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  12. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  13. 2024Diffusion Language Models Are Versatile Protein Learnerscited by 93
  14. 2023DiffuRec: A Diffusion Model for Sequential Recommendationcited by 91
  15. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  16. 2025A Survey on Diffusion Language Modelscited by 75
  17. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 73
  18. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  19. 2024Fisher Flow Matching for Generative Modeling over Discrete Datacited by 68
  20. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
  21. 2023CoCoGen: Physically-Consistent and Conditioned Score-based Generative Models for Forward and Inverse Problemscited by 54
  22. 2024Discrete Copula Diffusioncited by 53
  23. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  24. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 48
  25. 2023Diffusion Models for Non-autoregressive Text Generation: A Surveycited by 43
  26. 2022Empowering Diffusion Models on the Embedding Space for Text Generationcited by 40
  27. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39
  28. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 38
  29. 2024Full-Atom Peptide Design based on Multi-modal Flow Matchingcited by 34
  30. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  31. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Modelscited by 33
  32. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  33. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  34. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  35. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 31
  36. 2025TESS 2: A Large-Scale Generalist Diffusion Language Modelcited by 30
  37. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 30
  38. 2025Continuous Diffusion Model for Language Modelingcited by 29

50 most cited of 191 All 191 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Self-conditioned Embedding Diffusion for Text Generation
  2. 2025Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning
  3. 2025Understanding the Quality-Diversity Trade-off in Diffusion Language Models
  4. 2024Transfer Learning for Text Diffusion Models
  5. 2026Differences in Text Generated by Diffusion and Autoregressive Language Models
  6. 2024TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings
  7. 2026Diffusion Language Models: An Experimental Analysis
  8. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation