# dLLM: Simple Diffusion Language Modeling arXiv: https://arxiv.org/abs/2602.22661 ยท HTML: https://mlsanity.com/paper/2602.22661/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. - Builds on 56 of its 63 references that are in the map. - First posted on arXiv 26 Feb 2026. ## Placed in - Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/) ## Builds on (most cited first, 30 of 56) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021) - [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024) - [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019) - [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021) - [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023) - [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022) - [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021) - [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025) - [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019) - [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024) - [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022) - [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025) - [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024) - [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023) - [PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel](https://mlsanity.com/paper/2304.11277/) (2023) - [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024) - [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021) - [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024) - [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025) - [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022) - [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022) ## Built on by (most cited first, 14 of 14) - [DMax: Aggressive Parallel Decoding for dLLMs](https://mlsanity.com/paper/2604.08302/) (2026) - [Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation](https://mlsanity.com/paper/2606.06712/) (2026) - [When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models](https://mlsanity.com/paper/2604.23994/) (2026) - [Diffusion Language Models Are Natively Length-Aware](https://mlsanity.com/paper/2603.06123/) (2026) - [Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion](https://mlsanity.com/paper/2605.12825/) (2026) - [Continuous Language Diffusion as a Decoder-Interface Problem](https://mlsanity.com/paper/2606.08810/) (2026) - [DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech](https://mlsanity.com/paper/2607.04140/) (2026) - [CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Models](https://mlsanity.com/paper/2607.24763/) (2026) - [DALM: A Domain-Algebraic Language Model via Three-Phase Structured Generation](https://mlsanity.com/paper/2604.15593/) (2026) - [Extracting Training Data from Diffusion Language Models via Infilling](https://mlsanity.com/paper/2605.24173/) (2026) - [Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Models](https://mlsanity.com/paper/2606.16700/) (2026) - [Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Models](https://mlsanity.com/paper/2609.37391/) (2026) - [Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models](https://mlsanity.com/paper/2610.00661/) (2026) - [Nash Equilibrium Text: A Game-Theoretic Decoding Framework for Text Generation](https://mlsanity.com/paper/2610.05817/) (2026) ## Abstract (arXiv) Although diffusion language models (DLMs) are evolving quickly, many recent models converge on a set of shared components. These components, however, are distributed across ad-hoc research codebases or lack transparent implementations, making them difficult to reproduce or extend. As the field accelerates, there is a clear need for a unified framework that standardizes these common components while remaining flexible enough to support new methods and architectures. To address this gap, we introduce dLLM, an open-source framework that unifies the core components of diffusion language modeling -- training, inference, and evaluation -- and makes them easy to customize for new designs. With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream through a standardized pipeline. The framework also provides minimal, reproducible recipes for building small DLMs from scratch with accessible compute, including converting any BERT-style encoder or autoregressive LM into a DLM. We also release the checkpoints of these small DLMs to make DLMs more accessible and accelerate future research.