mlsanityData to 7 Oct 2026

LLaDA2.0: Scaling Up Diffusion Language Models to 100B

arXiv 2512.15745 ↗First posted 10 Dec 2025cs.LG cs.AI cs.CLcited by 154

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

This paper presents LLaDA2.0 -- a tuple of discrete diffusion large language models (dLLM) scaling up to 100B total parameters through systematic conversion from auto-regressive (AR) models -- establishing a new paradigm for frontier-scale deployment. Instead of costly training from scratch, LLaDA2.0 upholds knowledge inheritance, progressive adaption and efficiency-aware design principle, and seamless converts a pre-trained AR model into dLLM with a novel 3-phase block-level WSD based training scheme: progressive increasing block-size in block diffusion (warm-up), large-scale full-sequence diffusion (stable) and reverting back to compact-size block diffusion (decay). Along with post-training alignment with SFT and DPO, we obtain LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B), two instruction-tuned Mixture-of-Experts (MoE) variants optimized for practical deployment. By preserving the advantages of parallel decoding, these models deliver superior performance and efficiency at the frontier scale. Both models were open-sourced.

Key facts. Cited by 154 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 59 of its 69 references that are in the map. First posted on arXiv 10 Dec 2025.

Builds on

59 of 69 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025TiDAR: Think in Diffusion, Talk in Autoregression12 Nov
  2. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models10 Oct
  3. 2025dInfer: An Efficient Inference Framework for Diffusion Language Models9 Oct
  4. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  5. 2025Training Optimal Large Diffusion Language Models28 Sep
  6. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models8 Sep
  7. 2025Dream-Coder 7B: An Open Diffusion Language Model for Code1 Sep
  8. 2025Dream 7B: Diffusion Large Language Models21 Aug
  9. 2025DPad: Efficient Diffusion Language Models with Suffix Dropout19 Aug
  10. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  11. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  12. 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey16 Jun
2 more
  1. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct

Other references · 45

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
33 more
  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  7. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  11. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  12. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  13. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  14. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  15. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  16. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  17. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 277
  18. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  19. 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 195
  20. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 186
  21. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 182
  22. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  23. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solverscited by 155
  24. 2025BIG-Bench Extra Hardcited by 123
  25. 2020OCNLI: Original Chinese Natural Language Inferencecited by 86
  26. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
  27. 2024KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Taskscited by 67
  28. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  29. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 57
  30. 2025Every Activation Boosted: Scaling General Reasoner to 1 Trillion Open Language Foundationcited by 32
  31. 2025WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-trainingcited by 21
  32. 2025CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generationcited by 14
  33. 2025HARDMath2: A Benchmark for Applied Mathematics Built by Students as Part of a Graduate Classcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 152.

154 papers.

Sort:

By area:

  1. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 48
  2. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inferencecited by 38
  3. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  4. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 19
  5. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  6. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  7. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 12
  8. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Modelscited by 12
  9. 2026dLLM: Simple Diffusion Language Modelingcited by 12
  10. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 12
  11. 2026Deferred Commitment Decoding for Diffusion Language Modelscited by 11
  12. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
38 more
  1. 2026Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Modelscited by 9
  2. 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrowcited by 8
  3. 2026Reasoning with Latent Tokens in Diffusion Language Modelscited by 8
  4. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Trainingcited by 8
  5. 2026LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Modelcited by 8
  6. 2026DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmaskingcited by 8
  7. 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Modelscited by 8
  8. 2026Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Modelscited by 7
  9. 2026Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Modelscited by 6
  10. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 6
  11. 2026Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusingcited by 6
  12. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 6
  13. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 6
  14. 2026The Design Space of Tri-Modal Masked Diffusion Modelscited by 6
  15. 2026Introspective Diffusion Language Modelscited by 6
  16. 2026DLLM Agent: See Farther, Run Fastercited by 5
  17. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculationcited by 5
  18. 2026DARE: Diffusion Large Language Models Alignment and Reinforcement Executorcited by 5
  19. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Modelscited by 5
  20. 2026GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Modelscited by 5
  21. 2026Dynamic Expert Sharing: Decoupling Memory from Parallelism in Mixture-of-Experts Diffusion LLMscited by 4
  22. 2026Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Modelscited by 4
  23. 2026Learning from the Self-future: On-policy Self-distillation for dLLMscited by 4
  24. 2026DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Modelscited by 4
  25. 2026Trace-Based On-Policy Distillation for Masked Diffusion Language Modelscited by 4
  26. 2025Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMscited by 3
  27. 2026Training-Trajectory-Aware Token Selectioncited by 3
  28. 2026Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variantscited by 3
  29. 2026MaDiS: Taming Masked Diffusion Language Models for Sign Language Generationcited by 3
  30. 2026FOCUS: DLLMs Know How to Tame Their Compute Boundcited by 3
  31. 2026FlashBlock: Attention Caching for Efficient Long-Context Block Diffusioncited by 3
  32. 2026MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Modelscited by 3
  33. 2026Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effectscited by 3
  34. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 3
  35. 2026ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Drivingcited by 3
  36. 2026BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuningcited by 3
  37. 2026Learnability-Informed Fine-Tuning of Diffusion Language Modelscited by 3
  38. 2026Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTScited by 3

50 most cited of 154 All 154 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Large Language Diffusion Models
  2. 2025Fast-dLLM v2: Efficient Block-Diffusion LLM
  3. 2026Diffusion Language Models: An Experimental Analysis
  4. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Model
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
  6. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
  7. 2025Sequential Diffusion Language Models
  8. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques