mlsanityData to 14 Oct 2026

A Survey on Diffusion Language Models

arXiv 2508.10875 ↗First posted 14 Aug 2025cs.CL cs.AI cs.LGcited by 78

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm. By generating tokens in parallel through an iterative denoising process, DLMs possess inherent advantages in reducing inference latency and capturing bidirectional context, thereby enabling fine-grained control over the generation process. While achieving a several-fold speed-up, recent advancements have allowed DLMs to show performance comparable to their autoregressive counterparts, making them a compelling choice for various natural language processing tasks. In this survey, we provide a holistic overview of the current DLM landscape. We trace its evolution and relationship with other paradigms, such as autoregressive and masked language models, and cover both foundational principles and state-of-the-art models. Our work offers an up-to-date, comprehensive taxonomy and an in-depth analysis of current techniques, from pre-training strategies to advanced post-training methods. Another contribution of this survey is a thorough review of DLM inference strategies and optimizations, including improvements in decoding parallelism, caching mechanisms, and generation quality. We also highlight the latest approaches to multimodal extensions of DLMs and delineate their applications across various practical scenarios. Furthermore, our discussion addresses the limitations and challenges of DLMs, including efficiency, long-sequence handling, and infrastructure requirements, while outlining future research directions to sustain progress in this rapidly evolving field. Project GitHub is available at https://github.com/VILA-Lab/Awesome-DLMs.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 156 of its 228 references that are in the map. First posted on arXiv 14 Aug 2025.

Builds on

156 of 228 references.

Closest ancestors · 55

Same direction, newest first

  1. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  2. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models7 Jul
  3. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  4. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models23 Jun
  5. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs17 Jun
  6. 2025The Diffusion Duality12 Jun
  7. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generation30 May
  8. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  9. 2025Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation24 May
  10. 2025Anchored Diffusion Language Model24 May
  11. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  12. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
38 more
  1. 2025CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generation20 May
  2. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  3. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  4. 2025Generalized Interpolating Discrete Diffusion6 Mar
  5. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  6. 2025TESS 2: A Large-Scale Generalist Diffusion Language Model19 Feb
  7. 2025Large Language Diffusion Models14 Feb
  8. 2025Non-Markovian Discrete Diffusion with Causal Language Models13 Feb
  9. 2024Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models15 Dec
  10. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  11. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
  12. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  13. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
  14. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct
  15. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  16. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  17. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  18. 2024Improved Paraphrase Generation via Controllable Latent Diffusion13 Apr
  19. 2024DiffusionDialog: A Diffusion Model for Diverse Dialog Generation with Latent Space10 Apr
  20. 2024Let's Rectify Step by Step: Improving Aspect-based Sentiment Analysis with Diffusion Models23 Feb
  21. 2024Transfer Learning for Text Diffusion Models30 Jan
  22. 2024ROIC-DM: Robust Text Inference and Classification via Diffusion Model7 Jan
  23. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  24. 2023InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation18 Oct
  25. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning23 Aug
  26. 2023XDLM: Cross-lingual Diffusion Language Model for Machine Translation25 Jul
  27. 2023PoetryDiffusion: Towards Joint Semantic and Metrical Manipulation in Poetry Generation14 Jun
  28. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model5 Jun
  29. 2023Likelihood-Based Diffusion Language Models30 May
  30. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation16 May
  31. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusion15 May
  32. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation6 May
  33. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  34. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise22 Dec
  35. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  36. 2022Latent Diffusion for Language Generation19 Dec
  37. 2022Empowering Diffusion Models on the Embedding Space for Text Generation19 Dec
  38. 2022Continuous diffusion for categorical data28 Nov

50 newest of 55 All 55 (JSON)

Other references · 101

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2024The Llama 3 Herd of Modelscited by 14,715
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  11. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  12. 2020Denoising Diffusion Implicit Modelscited by 9,781
38 more
  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  10. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  13. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  14. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  15. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  16. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  17. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  18. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  19. 2023Qwen Technical Reportcited by 3,344
  20. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  21. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  22. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  23. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  24. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  25. 2023A Survey of Large Language Modelscited by 2,576
  26. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  27. 2024Qwen2 Technical Reportcited by 2,120
  28. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  29. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  30. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  31. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  32. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  33. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  34. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  35. 2023Adversarial Diffusion Distillationcited by 759
  36. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  37. 2025Kimi K2: Open Agentic Intelligencecited by 567
  38. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 24; 2026: 54.

78 papers.

Sort:

By area:

  1. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 129
  2. 2026ELF: Embedded Language Flowscited by 45
  3. 2025Scaling Behavior of Discrete Diffusion Language Modelscited by 35
  4. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  5. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 25
  6. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 23
  7. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  8. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Modelscited by 17
  9. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Thinkcited by 14
  10. 2025UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Modelscited by 13
  11. 2025Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Modelcited by 13
  12. 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Modelscited by 12
38 more
  1. 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrowcited by 11
  2. 2025Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerabilitycited by 10
  3. 2025LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendationcited by 10
  4. 2025Deep Thinking by Markov Chain of Continuous Thoughtscited by 9
  5. 2026Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Modelscited by 9
  6. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decodingcited by 7
  7. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  8. 2025DiffuMamba: High-Throughput Diffusion LMs with Mamba Backbonecited by 6
  9. 2025Decoding Large Language Diffusion Models with Foreseeing Movementcited by 6
  10. 2026FLARE: Diffusion for Hybrid Language Modelcited by 6
  11. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Modelscited by 5
  12. 2025From Masks to Worlds: A Hitchhiker's Guide to World Modelscited by 5
  13. 2025Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Modelscited by 4
  14. 2025WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoningcited by 4
  15. 2026Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inferencecited by 4
  16. 2025Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoningcited by 3
  17. 2025Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibriumcited by 3
  18. 2026Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Modelscited by 3
  19. 2026Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Modelscited by 3
  20. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Modelscited by 3
  21. 2026DICE: Diffusion Large Language Models Excel at Generating CUDA Kernelscited by 3
  22. 2026Few-Step Diffusion Language Models via Trajectory Self-Distillationcited by 3
  23. 2026MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Modelscited by 3
  24. 2026Generalized Discrete Diffusion from Snapshotscited by 3
  25. 2026Dataset-Level Metrics Attenuate Non-Determinism: A Fine-Grained Non-Determinism Evaluation in Diffusion Language Modelscited by 3
  26. 2026Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokenscited by 3
  27. 2025Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentationcited by 2
  28. 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Cachingcited by 2
  29. 2026Sink-Aware Pruning for Diffusion Language Modelscited by 2
  30. 2026OSCAR: Orchestrated Self-verification and Cross-path Refinementcited by 2
  31. 2026Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Modelscited by 2
  32. 2026TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLMcited by 2
  33. 2026TrajDLM: Topology-Aware Block Diffusion Language Model for Trajectory Generationcited by 2
  34. 2026dMoE: dLLMs with Learnable Block Expertscited by 2
  35. 2026Rethinking the Generation Order of Block Diffusion Language Modelscited by 2
  36. 2025C2DLM: Causal Concept-Guided Diffusion Large Language Modelscited by 1
  37. 2026dgMARK: Decoding-Guided Watermarking for Diffusion Language Modelscited by 1
  38. 2026Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equationscited by 1

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Language Models: An Experimental Analysis
  2. 2026Revisiting the 16x Pretraining Compute Gap between Diffusion and Autoregressive Language Models
  3. 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey
  4. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  5. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
  6. 2026dLLM: Simple Diffusion Language Modeling
  7. 2026Introspective Diffusion Language Models
  8. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices