mlsanityData to 7 Oct 2026

Dream 7B: Diffusion Large Language Models

arXiv 2508.15487 ↗First posted 21 Aug 2025cs.CLcited by 515

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

We introduce Dream 7B, the most powerful open diffusion large language model to date. Unlike autoregressive (AR) models that generate tokens sequentially, Dream 7B employs discrete diffusion modeling to refine sequences in parallel through iterative denoising. Our model consistently outperforms existing diffusion language models on general, mathematical, and coding tasks. Dream 7B demonstrates superior planning abilities and inference flexibility, including arbitrary-order generation, infilling capabilities, and tunable quality-speed trade-offs. These results are achieved through simple yet effective training techniques, including AR-based LLM initialization and context-adaptive token-level noise rescheduling. We release both Dream-Base and Dream-Instruct to facilitate further research in diffusion-based language modeling.

Key facts. Cited by 515 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 46 of its 55 references that are in the map. First posted on arXiv 21 Aug 2025.

Builds on

46 of 55 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  2. 2025Implicit Search via Discrete Diffusion: A Study on Chess27 Feb
  3. 2025Large Language Diffusion Models14 Feb
  4. 2024Scaling up Masked Diffusion Models on Text24 Oct
  5. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  6. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  10. 2023DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models9 Oct
  11. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning23 Aug
  12. 2023Likelihood-Based Diffusion Language Models30 May
5 more
  1. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  2. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise22 Dec
  3. 2022Continuous diffusion for categorical data28 Nov
  4. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  5. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  12. 2021Program Synthesis with Large Language Modelscited by 3,509
17 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  2. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2025s1: Simple test-time scalingcited by 1,311
  7. 2021Variational Diffusion Modelscited by 1,145
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  9. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  10. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  11. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  12. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  13. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  14. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  15. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  16. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  17. 2024The pitfalls of next-token predictioncited by 156

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 112; 2026: 403.

515 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  2. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  3. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  4. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  5. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  6. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  7. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 54
  8. 2025Diffusion Language Models are Super Data Learnerscited by 45
  9. 2025DiffuSpec: Unlocking Diffusion Language Models for Speculative Decodingcited by 40
  10. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inferencecited by 38
  11. 2025TiDAR: Think in Diffusion, Talk in Autoregressioncited by 35
  12. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
38 more
  1. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  2. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 31
  3. 2025LLaDA-VLA: Vision Language Diffusion Action Modelscited by 30
  4. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 30
  5. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 30
  6. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 30
  7. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  8. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  9. 2026LangFlow: Continuous Diffusion Rivals Discrete in Language Modelingcited by 28
  10. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimationcited by 27
  11. 2026ELF: Embedded Language Flowscited by 27
  12. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 26
  13. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  14. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 24
  15. 2026DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inferencecited by 23
  16. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Modelscited by 22
  17. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 22
  18. 2025Training Optimal Large Diffusion Language Modelscited by 21
  19. 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policiescited by 21
  20. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagationcited by 21
  21. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 20
  22. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 20
  23. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  24. 2025OBS-Diff: Accurate Pruning For Diffusion Models in One-Shotcited by 19
  25. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 19
  26. 2025NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matchingcited by 19
  27. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 19
  28. 2025Attention Sinks in Diffusion Language Modelscited by 18
  29. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  30. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 17
  31. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  32. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  33. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Modelscited by 16
  34. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Stepcited by 16
  35. 2025SparseD: Sparse Attention for Diffusion Language Modelscited by 16
  36. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decodingcited by 16
  37. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Modelscited by 16
  38. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 16

50 most cited of 515 All 515 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  2. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  3. 2025A Survey on Diffusion Language Models
  4. 2025Unifying Autoregressive and Diffusion-Based Sequence Generation
  5. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
  6. 2025Sequential Diffusion Language Models
  7. 2026Scaling Beyond Masked Diffusion Language Models
  8. 2025On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond