mlsanityData to 7 Oct 2026

d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation

arXiv 2601.07568 ↗First posted 12 Jan 2026cs.LG cs.AIcited by 34

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion large language models (dLLMs) offer capabilities beyond those of autoregressive (AR) LLMs, such as parallel decoding and random-order generation. However, realizing these benefits in practice is non-trivial, as dLLMs inherently face an accuracy-parallelism trade-off. Despite increasing interest, existing methods typically focus on only one-side of the coin, targeting either efficiency or accuracy. To address this limitation, we propose d3LLM (Pseudo-Distilled Diffusion Large Language Model), striking a balance between accuracy and parallelism: (i) during training, we introduce pseudo-trajectory distillation to teach the model which tokens can be decoded confidently at early steps, thereby improving parallelism; (ii) during inference, we employ entropy-based multi-block decoding with a KV-cache refresh mechanism to achieve high parallelism while maintaining accuracy. To better evaluate dLLMs, we also introduce AUP (Accuracy Under Parallelism), a new metric that jointly measures accuracy and parallelism. Experiments demonstrate that our d3LLM achieves up to 10× speedup over vanilla LLaDA/Dream, and 5× speedup over AR models without much accuracy drop. Our code is available at https://github.com/hao-ai-lab/d3LLM.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 54 of its 65 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

54 of 65 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding18 Dec
  2. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcing16 Dec
  3. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding15 Dec
  4. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs6 Oct
  5. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models30 Sep
  6. 2025dParallel: Learnable Parallel Decoding for dLLMs30 Sep
  7. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step28 Sep
  8. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
  10. 2025dKV-Cache: The Cache for Diffusion Language Models21 May

Other references · 44

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  10. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  11. 2025Large Language Diffusion Modelscited by 872
  12. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
32 more
  1. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  2. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  3. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  4. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  5. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  6. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  7. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 295
  8. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  10. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  11. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
  12. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  13. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  14. 2023REST: Retrieval-Based Speculative Decodingcited by 148
  15. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  16. 2025The Diffusion Dualitycited by 124
  17. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  18. 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decodingcited by 113
  19. 2023Online Speculative Decodingcited by 109
  20. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  21. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  22. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 80
  23. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 73
  24. 2024CLLMs: Consistency Large Language Modelscited by 71
  25. 2021Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimizationcited by 66
  26. 2025dInfer: An Efficient Inference Framework for Diffusion Language Modelscited by 33
  27. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 30
  28. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 29
  29. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  30. 2024PLD+: Accelerating LLM inference by leveraging Language Model Artifactscited by 16
  31. 2025TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Treecited by 11
  32. 2025Provably Efficient Online RLHF with One-Pass Reward Modelingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 34.

34 papers.

Sort:

By area:

  1. 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 120
  2. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 19
  3. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  4. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMscited by 12
  5. 2026JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Draftingcited by 12
  6. 2026Improved Large Language Diffusion Modelscited by 12
  7. 2026Triplets Better Than Pairs: Towards Stable and Effective Self-Play Fine-Tuning for LLMscited by 9
  8. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Accelerationcited by 8
  9. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  10. 2026Learning from the Self-future: On-policy Self-distillation for dLLMscited by 4
  11. 2026Few-Step Diffusion Language Models via Trajectory Self-Distillationcited by 3
  12. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 3
22 more
  1. 2026OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Modelscited by 3
  2. 2026A Comparative analysis of Layer-wise Representational Capacity in AR and Diffusion LLMscited by 2
  3. 2026TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLMcited by 2
  4. 2026Multi-Block Diffusion Language Modelscited by 2
  5. 2026TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decodingcited by 2
  6. 2026Generalized Discrete Diffusion with Self-Correctioncited by 1
  7. 2026TRIMS: Trajectory-Ranked Instruction Masked Supervision for Diffusion Language Modelscited by 1
  8. 2026Supportive Token Revealing for Fast Diffusion Language Model Decodingcited by 1
  9. 2026Enabling KV Caching of Shared Prefix for Diffusion Language Modelscited by 1
  10. 2026Efficient On-Device Diffusion LLM Inference with Mobile NPUcited by 1
  11. 2026PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Modelscited by 1
  12. 2026CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Modelscited by 1
  13. 2026CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcingcited by 1
  14. 2026AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decodingcited by 0
  15. 2026Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioningcited by 0
  16. 2026Accelerating Diffusion Language Models via Structured Suffix Modelingcited by 0
  17. 2026CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Modelscited by 0
  18. 2026Retrospective Distillation Attribution via Normalized Response Similaritycited by 0
  19. 2026CT-OPD: Counterfactual Trace On-Policy Distillation for Diffusion Vision-Language Modelscited by 0
  20. 2026Faster Block-Diffusion Serving with Distribution-Free Risk Guaranteescited by 0
  21. 2026d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Modelscited by 0
  22. 2026STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  2. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  3. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagation
  4. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding
  5. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
  6. 2025Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
  7. 2025Sequential Diffusion Language Models
  8. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models