mlsanityData to 7 Oct 2026

Latent Diffusion for Language Generation

arXiv 2212.09462 ↗First posted 19 Dec 2022cs.CL cs.LGcited by 110

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion models have achieved great success in modeling continuous data modalities such as images, audio, and video, but have seen limited use in discrete domains such as language. Recent attempts to adapt diffusion to language have presented diffusion as an alternative to existing pretrained language models. We view diffusion and existing language models as complementary. We demonstrate that encoder-decoder language models can be utilized to efficiently learn high-quality language autoencoders. We then demonstrate that continuous diffusion models can be learned in the latent space of the language autoencoder, enabling us to sample continuous latent representations that can be decoded into natural language with the pretrained decoder. We validate the effectiveness of our approach for unconditional, class-conditional, and sequence-to-sequence language generation. We demonstrate across multiple diverse data sets that our latent language diffusion models are significantly more effective than previous diffusion language models.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 57 of its 81 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

57 of 81 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Continuous diffusion for categorical data28 Nov
  2. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  3. 2022Self-conditioned Embedding Diffusion for Text Generation8 Nov
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct
  5. 2021Autoregressive Diffusion Models5 Oct

Other references · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2016Densely Connected Convolutional Networkscited by 6,691
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
38 more
  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2016Layer Normalizationcited by 5,579
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  12. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  13. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  14. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  15. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  16. 2020GLU Variants Improve Transformercited by 1,724
  17. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  18. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  19. 2019Root Mean Square Layer Normalizationcited by 1,425
  20. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  21. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  22. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  23. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  24. 2021Variational Diffusion Modelscited by 1,145
  25. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  26. 2020On Layer Normalization in the Transformer Architecturecited by 906
  27. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  28. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  29. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  30. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  31. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  32. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  33. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  34. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  35. 2021True Few-Shot Learning with Language Modelscited by 343
  36. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  37. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  38. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 27; 2025: 30; 2026: 40.

110 papers.

Sort:

By area:

  1. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  2. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  3. 2024Discrete Flow Matchingcited by 301
  4. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  5. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  6. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  7. 2023Likelihood-Based Diffusion Language Modelscited by 147
  8. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  9. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
  10. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  11. 2025A Survey on Diffusion Language Modelscited by 75
  12. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 73
38 more
  1. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  2. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  3. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Modelcited by 56
  4. 2024Discrete Copula Diffusioncited by 53
  5. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  6. 2023Diffusion Models for Non-autoregressive Text Generation: A Surveycited by 43
  7. 2024Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editingcited by 38
  8. 2025Efficient Diffusion Models: A Surveycited by 35
  9. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 31
  10. 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 30
  11. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  12. 2026ELF: Embedded Language Flowscited by 27
  13. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  14. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modelingcited by 24
  15. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  16. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generationcited by 19
  17. 2023Diffusion Models in NLP: A Surveycited by 16
  18. 2023A Survey of Diffusion Models in Natural Language Processingcited by 16
  19. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Modelscited by 16
  20. 2025Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradientscited by 15
  21. 2024CTSyn: A Foundation Model for Cross Tabular Data Generationcited by 13
  22. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Languagecited by 13
  23. 2024DiscDiff: Latent Diffusion Model for DNA Sequence Generationcited by 12
  24. 2024Diffusion on language model encodings for protein sequence generationcited by 11
  25. 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
  26. 2024TimeLDM: Latent Diffusion Model for Unconditional Time Series Generationcited by 10
  27. 2024Decoupling Training-Free Guided Diffusion by ADMMcited by 10
  28. 2025dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learningcited by 10
  29. 2025What Exactly Does Guidance Do in Masked Discrete Diffusion Modelscited by 9
  30. 2025DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretationcited by 9
  31. 2024Preference Diffusion for Recommendationcited by 8
  32. 2024DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreakcited by 8
  33. 2025Smoothie: Smoothing Diffusion on Token Embeddings for Text Generationcited by 8
  34. 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 8
  35. 2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Spacecited by 8
  36. 2026Context Tokens are Anchors: Understanding the Repetition Curse in dMLLMs from an Information Flow Perspectivecited by 6
  37. 2025DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Modelscited by 5
  38. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Modelscited by 5

50 most cited of 110 All 110 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
  2. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
  3. 2025Latent-Augmented Discrete Diffusion Models
  4. 2026Diffusion Language Models: An Experimental Analysis
  5. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  6. 2024Multimodal Latent Language Modeling with Next-Token Diffusion
  7. 2025Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning
  8. 2026CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language