mlsanityData to 7 Oct 2026

Fast-dLLM v2: Efficient Block-Diffusion LLM

arXiv 2509.26328 ↗First posted 30 Sep 2025cs.CLcited by 128

Not on the map. Why?

Cited by 78 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent sequential decoding limits inference efficiency. In this work, we propose Fast-dLLM v2, a carefully designed block diffusion language model (dLLM) that efficiently adapts pretrained AR models into dLLMs for parallel text generation, requiring only approximately 1B tokens of fine-tuning. This represents a 500x reduction in training data compared to full-attention diffusion LLMs such as Dream (580B tokens), while preserving the original model's performance. Our approach introduces a novel training recipe that combines a block diffusion mechanism with a complementary attention mask, enabling blockwise bidirectional context modeling without sacrificing AR training objectives. To further accelerate decoding, we design a hierarchical caching mechanism: a block-level cache that stores historical context representations across blocks, and a sub-block cache that enables efficient parallel generation within partially decoded blocks. Coupled with our parallel decoding pipeline, Fast-dLLM v2 achieves up to 2.5x speedup over standard AR decoding without compromising generation quality. Extensive experiments across diverse benchmarks demonstrate that Fast-dLLM v2 matches or surpasses AR baselines in accuracy, while delivering state-of-the-art efficiency among dLLMs - marking a significant step toward the practical deployment of fast and accurate LLMs. Code and model will be publicly released.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

26 of 35 references.

References · 26

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  3. 2025Large Language Diffusion Modelscited by 872
  4. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  5. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  6. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  7. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  8. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  9. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  10. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  11. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  12. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
14 more
  1. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  2. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 140
  3. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  4. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  6. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  7. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
  8. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  9. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 64
  10. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 45
  11. 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 43
  12. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 38
  13. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Modelscited by 33
  14. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 114.

128 papers.

Sort:

By area:

  1. 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 120
  2. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  3. 2025Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Waycited by 22
  4. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inferencecited by 21
  5. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 20
  6. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 20
  7. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 19
  8. 2025Attention Sinks in Diffusion Language Modelscited by 18
  9. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  10. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  11. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  12. 2025SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understandingcited by 12
38 more
  1. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 12
  2. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMscited by 12
  3. 2026dLLM: Simple Diffusion Language Modelingcited by 12
  4. 2026JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Draftingcited by 12
  5. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcingcited by 11
  6. 2025Corrective Diffusion Language Modelscited by 11
  7. 2026Deferred Commitment Decoding for Diffusion Language Modelscited by 11
  8. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
  9. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practicescited by 10
  10. 2025DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Modelscited by 10
  11. 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Modelscited by 10
  12. 2026Residual Context Diffusion Language Modelscited by 9
  13. 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusioncited by 9
  14. 2026Stabilizing Reinforcement Learning for Diffusion Language Modelscited by 9
  15. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Accelerationcited by 8
  16. 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 8
  17. 2026DSB: Dynamic Sliding Block Scheduling for Diffusion LLMscited by 7
  18. 2026DLLM-Searcher: Adapting Diffusion Large Language Model for Search Agentscited by 7
  19. 2026MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decodingcited by 7
  20. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  21. 2026xPress: Parallel Refinement for Diffusion Drafters in Speculative Decodingcited by 7
  22. 2026Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Modelscited by 6
  23. 2026The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Checkcited by 6
  24. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decodingcited by 6
  25. 2026Improving Sampling for Masked Diffusion Models via Information Gaincited by 6
  26. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  27. 2026DLLM Agent: See Farther, Run Fastercited by 5
  28. 2026Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Modelscited by 4
  29. 2026DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Modelscited by 4
  30. 2025Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMscited by 3
  31. 2026Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variantscited by 3
  32. 2026FOCUS: DLLMs Know How to Tame Their Compute Boundcited by 3
  33. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Modelscited by 3
  34. 2026Few-Step Diffusion Language Models via Trajectory Self-Distillationcited by 3
  35. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 3
  36. 2026EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Modelscited by 3
  37. 2026dTRPO: Trajectory Reduction in Policy Optimization of Diffusion Large Language Modelscited by 3
  38. 2026Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Modelscited by 3

50 most cited of 128 All 128 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
  2. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size
  3. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  4. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  5. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
  6. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
  7. 2025Sequential Diffusion Language Models
  8. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching