mlsanityData to 7 Oct 2026

dLLM: Simple Diffusion Language Modeling

arXiv 2602.22661 ↗First posted 26 Feb 2026cs.CL cs.AI cs.LGcited by 12

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Although diffusion language models (DLMs) are evolving quickly, many recent models converge on a set of shared components. These components, however, are distributed across ad-hoc research codebases or lack transparent implementations, making them difficult to reproduce or extend. As the field accelerates, there is a clear need for a unified framework that standardizes these common components while remaining flexible enough to support new methods and architectures. To address this gap, we introduce dLLM, an open-source framework that unifies the core components of diffusion language modeling -- training, inference, and evaluation -- and makes them easy to customize for new designs. With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream through a standardized pipeline. The framework also provides minimal, reproducible recipes for building small DLMs from scratch with accessible compute, including converting any BERT-style encoder or autoregressive LM into a DLM. We also release the checkpoints of these small DLMs to make DLMs more accessible and accelerate future research.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 56 of its 63 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

56 of 63 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed16 Dec
  2. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  3. 2025TiDAR: Think in Diffusion, Talk in Autoregression12 Nov
  4. 2025Planned Diffusion20 Oct
  5. 2025Dream 7B: Diffusion Large Language Models21 Aug
  6. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  7. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  8. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  9. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  10. 2025Large Language Diffusion Models14 Feb
  11. 2024Scaling up Masked Diffusion Models on Text24 Oct
  12. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
8 more
  1. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  2. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  4. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  6. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise22 Dec
  7. 2022Continuous diffusion for categorical data28 Nov
  8. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  11. 2024Qwen2.5 Technical Reportcited by 4,713
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
24 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,509
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  5. 2025s1: Simple test-time scalingcited by 1,311
  6. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  7. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  8. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  9. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  10. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  11. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  13. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  14. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
  15. 2024Discrete Flow Matchingcited by 301
  16. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  17. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  18. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 167
  19. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  20. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  21. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  22. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  23. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusioncited by 57
  24. 2025OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flowscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  2. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
  3. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  4. 2026Diffusion Language Models Are Natively Length-Awarecited by 5
  5. 2026Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusioncited by 3
  6. 2026DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speechcited by 1
  7. 2026CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Modelscited by 1
  8. 2026DALM: A Domain-Algebraic Language Model via Three-Phase Structured Generationcited by 0
  9. 2026Extracting Training Data from Diffusion Language Models via Infillingcited by 0
  10. 2026Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Modelscited by 0
  11. 2026Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Modelscited by 0
  12. 2026Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Diffusion Language Models
  2. 2026Diffusion Language Models: An Experimental Analysis
  3. 2026Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design
  4. 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey
  5. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
  6. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
  7. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  8. 2025Sequential Diffusion Language Models