mlsanityData to 14 Oct 2026

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

arXiv 2210.08933 ↗First posted 17 Oct 2022cs.CL cs.LGcited by 450

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusion models to natural language is under-explored due to the discrete nature of texts, especially for conditional generation. We tackle this challenge by proposing DiffuSeq: a diffusion model designed for sequence-to-sequence (Seq2Seq) text generation tasks. Upon extensive evaluation over a wide range of Seq2Seq tasks, we find DiffuSeq achieving comparable or even better performance than six established baselines, including a state-of-the-art model that is based on pre-trained language models. Apart from quality, an intriguing property of DiffuSeq is its high diversity during generation, which is desired in many Seq2Seq tasks. We further include a theoretical analysis revealing the connection between DiffuSeq and autoregressive/non-autoregressive models. Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks. Code is available at urlhttps://github.com/Shark-NLP/DiffuSeq

Key facts. Cited by 450 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 43 of its 53 references that are in the map. First posted on arXiv 17 Oct 2022.

Builds on

43 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modeling13 Jun

Other references · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  11. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  12. 2013Auto-Encoding Variational Bayescited by 4,053
30 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  3. 2016Improved Techniques for Training GANscited by 2,970
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2016Density estimation using Real NVPcited by 2,042
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  8. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  9. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  10. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  12. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  13. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 582
  14. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  15. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  16. 2017Non-Autoregressive Neural Machine Translationcited by 463
  17. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
  18. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  19. 2017Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoderscited by 222
  20. 2016Unrolled Generative Adversarial Networkscited by 208
  21. 2019Levenshtein Transformercited by 199
  22. 2019Well-Read Students Learn Better: On the Importance of Pre-training Compact Modelscited by 120
  23. 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
  24. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  25. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  26. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 96
  27. 2018Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speechcited by 55
  28. 2017Quasar: Datasets for Question Answering by Search and Readingcited by 50
  29. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  30. 2022Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priorscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 12; 2023: 95; 2024: 80; 2025: 131; 2026: 132.

450 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 912
  2. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  3. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  4. 2023DeepCache: Accelerating Diffusion Models for Freecited by 359
  5. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  6. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  7. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  8. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  9. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 246
  10. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  11. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 224
  12. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 221
38 more
  1. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  2. 2024Scaling up Masked Diffusion Models on Textcited by 199
  3. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  4. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  5. 2024The pitfalls of next-token predictioncited by 160
  6. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  7. 2023Likelihood-Based Diffusion Language Modelscited by 151
  8. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  9. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  10. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 130
  11. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 129
  12. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 118
  13. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillationcited by 116
  14. 2022Latent Diffusion for Language Generationcited by 115
  15. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 115
  16. 2023SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillationcited by 110
  17. 2024Diffusion Language Models Are Versatile Protein Learnerscited by 98
  18. 2023DiffuRec: A Diffusion Model for Sequential Recommendationcited by 93
  19. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  20. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
  21. 2023Non-autoregressive Conditional Diffusion Models for Time Series Predictioncited by 78
  22. 2025A Survey on Diffusion Language Modelscited by 78
  23. 2023On the Generalization Properties of Diffusion Modelscited by 68
  24. 2023Priority-Centric Human Motion Generation in Discrete Latent Spacecited by 67
  25. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 67
  26. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 66
  27. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 65
  28. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 64
  29. 2023One-Step Diffusion Distillation via Deep Equilibrium Modelscited by 61
  30. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  31. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Modelcited by 58
  32. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 57
  33. 2023Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimizationcited by 56
  34. 2023CoCoGen: Physically-Consistent and Conditioned Score-based Generative Models for Forward and Inverse Problemscited by 56
  35. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 56
  36. 2023DiffusionRet: Generative Text-Video Retrieval with Diffusion Modelcited by 52
  37. 2023Spatio-temporal Diffusion Point Processescited by 52
  38. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50

50 most cited of 450 All 450 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022DiffusER: Discrete Diffusion via Edit-based Reconstruction
  2. 2025SA-DiffuSeq: Addressing Computational and Scalability Challenges in Long-Document Generation with Sparse Attention
  3. 2022Self-conditioned Embedding Diffusion for Text Generation
  4. 2024Energy-Based Diffusion Language Models for Text Generation
  5. 2025Theoretical Benefit and Limitation of Diffusion Language Model
  6. 2025Discrete Diffusion Models for Language Generation
  7. 2025Unifying Autoregressive and Diffusion-Based Sequence Generation
  8. 2025MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Experts