# Fast-dLLM v2: Efficient Block-Diffusion LLM

arXiv: https://arxiv.org/abs/2509.26328 · HTML: https://mlsanity.com/paper/2509.26328/

## Key facts

- Cited by 139 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 26 of 26)

- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025)
- [Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models](https://mlsanity.com/paper/2503.09573/) (2025)
- [LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models](https://mlsanity.com/paper/2505.19223/) (2025)
- [SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control](https://mlsanity.com/paper/2210.17432/) (2022)
- [dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching](https://mlsanity.com/paper/2506.06295/) (2025)
- [dKV-Cache: The Cache for Diffusion Language Models](https://mlsanity.com/paper/2505.15781/) (2025)
- [Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking](https://mlsanity.com/paper/2505.24857/) (2025)
- [AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation](https://mlsanity.com/paper/2305.09515/) (2023)
- [Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing](https://mlsanity.com/paper/2508.09192/) (2025)
- [Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding](https://mlsanity.com/paper/2505.16990/) (2025)
- [LaViDa: A Large Diffusion Language Model for Multimodal Understanding](https://mlsanity.com/paper/2505.16839/) (2025)
- [Diffusion Language Models Know the Answer Before Decoding](https://mlsanity.com/paper/2508.19982/) (2025)
- [Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction](https://mlsanity.com/paper/2508.02558/) (2025)
- [Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs](https://mlsanity.com/paper/2507.18578/) (2025)
- [Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles](https://mlsanity.com/paper/2506.10848/) (2025)
- [DPad: Efficient Diffusion Language Models with Suffix Dropout](https://mlsanity.com/paper/2508.14148/) (2025)
- [Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models](https://mlsanity.com/paper/2508.09138/) (2025)
- [Set Block Decoding is a Language Model Inference Accelerator](https://mlsanity.com/paper/2509.04185/) (2025)

## Built on by (most cited first, 30 of 139)

- [DFlash: Block Diffusion for Flash Speculative Decoding](https://mlsanity.com/paper/2602.06036/) (2026)
- [d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation](https://mlsanity.com/paper/2601.07568/) (2026)
- [LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding](https://mlsanity.com/paper/2605.27365/) (2026)
- [Encoder-Decoder Diffusion Language Models for Efficient Training and Inference](https://mlsanity.com/paper/2510.22852/) (2025)
- [Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding](https://mlsanity.com/paper/2607.05722/) (2026)
- [Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Way](https://mlsanity.com/paper/2510.24605/) (2025)
- [CDLM: Consistency Diffusion Language Models For Faster Sampling](https://mlsanity.com/paper/2511.19269/) (2025)
- [Attention Sinks in Diffusion Language Models](https://mlsanity.com/paper/2510.15731/) (2025)
- [Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone](https://mlsanity.com/paper/2512.22615/) (2025)
- [DMax: Aggressive Parallel Decoding for dLLMs](https://mlsanity.com/paper/2604.08302/) (2026)
- [JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting](https://mlsanity.com/paper/2606.18394/) (2026)
- [dLLM: Simple Diffusion Language Modeling](https://mlsanity.com/paper/2602.22661/) (2026)
- [HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better](https://mlsanity.com/paper/2607.04884/) (2026)
- [SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understanding](https://mlsanity.com/paper/2512.14068/) (2025)
- [Fast and Accurate Causal Parallel Decoding using Jacobi Forcing](https://mlsanity.com/paper/2512.14681/) (2025)
- [Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model](https://mlsanity.com/paper/2601.15892/) (2026)
- [DAWN: Dependency-Aware Fast Inference for Diffusion LLMs](https://mlsanity.com/paper/2602.06953/) (2026)
- [Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion](https://mlsanity.com/paper/2603.06577/) (2026)
- [Deferred Commitment Decoding for Diffusion Language Models](https://mlsanity.com/paper/2601.02076/) (2026)
- [Corrective Diffusion Language Models](https://mlsanity.com/paper/2512.15596/) (2025)
- [Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation](https://mlsanity.com/paper/2606.06712/) (2026)
- [DiffusionGemma Technical Report](https://mlsanity.com/paper/2608.00146/) (2026)
- [How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices](https://mlsanity.com/paper/2510.18480/) (2025)
- [DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models](https://mlsanity.com/paper/2512.15713/) (2025)
- [DiRL: An Efficient Post-Training Framework for Diffusion Language Models](https://mlsanity.com/paper/2512.22234/) (2025)
- [Residual Context Diffusion Language Models](https://mlsanity.com/paper/2601.22954/) (2026)
- [Stabilizing Reinforcement Learning for Diffusion Language Models](https://mlsanity.com/paper/2603.06743/) (2026)
- [Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM](https://mlsanity.com/paper/2604.06832/) (2026)
- [DLLM-Searcher: Adapting Diffusion Large Language Model for Search Agents](https://mlsanity.com/paper/2602.07035/) (2026)
- [TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration](https://mlsanity.com/paper/2602.08404/) (2026)

## Abstract (arXiv)

Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent sequential decoding limits inference efficiency. In this work, we propose Fast-dLLM v2, a carefully designed block diffusion language model (dLLM) that efficiently adapts pretrained AR models into dLLMs for parallel text generation, requiring only approximately 1B tokens of fine-tuning. This represents a 500x reduction in training data compared to full-attention diffusion LLMs such as Dream (580B tokens), while preserving the original model's performance. Our approach introduces a novel training recipe that combines a block diffusion mechanism with a complementary attention mask, enabling blockwise bidirectional context modeling without sacrificing AR training objectives. To further accelerate decoding, we design a hierarchical caching mechanism: a block-level cache that stores historical context representations across blocks, and a sub-block cache that enables efficient parallel generation within partially decoded blocks. Coupled with our parallel decoding pipeline, Fast-dLLM v2 achieves up to 2.5x speedup over standard AR decoding without compromising generation quality. Extensive experiments across diverse benchmarks demonstrate that Fast-dLLM v2 matches or surpasses AR baselines in accuracy, while delivering state-of-the-art efficiency among dLLMs - marking a significant step toward the practical deployment of fast and accurate LLMs. Code and model will be publicly released.