mlsanityData to 7 Oct 2026

TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLM

arXiv 2605.09536 ↗First posted 10 May 2026cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Diffusion large language models (dLLMs) offer a promising paradigm for parallel text generation, but in practice they face an accuracy-parallelism trade-off, where increasing tokens per forward (TPF) often degrades generation quality. Existing acceleration methods often gain speed at the cost of accuracy. To address this limitation, we propose TAD, a Temporal-Aware trajectory self-Distillation framework. During data construction, we condition a teacher model on both the prompt and the ground-truth response to generate decoding trajectories, recording the intermediate masked states throughout the process. Based on how many decoding steps remain before each masked token is revealed, we partition masked positions into near and distant subsets. For near tokens, we train the student with a hard cross-entropy loss using the teacher trajectory tokens as labels, encouraging confident predictions for tokens that are about to be decoded. For distant tokens, we apply a soft KL divergence loss between the teacher and student token distributions, providing softer supervision and preserving future planning knowledge. This temporal-aware partition naturally gives rise to two deployment configurations: a Quality model that prioritizes accuracy and a Speed model that favors more aggressive acceleration. Experiments show that TAD consistently improves the accuracy-parallelism trade-off. On LLaDA, it raises average accuracy from 46.2% to 51.6% with the Quality model and average AUP from 46.2 to 257.1 with the Speed model. Our code is available at: https://github.com/BHmingyang/TAD

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 49 of its 51 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

49 of 51 references.

References · 49

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2022Diffusion Models in Vision: A Surveycited by 1,008
  10. 2025Large Language Diffusion Modelscited by 872
  11. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  12. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
37 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  2. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  3. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  4. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  5. 2026Reinforcement Learning via Self-Distillationcited by 285
  6. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  7. 2026Self-Distillation Enables Continual Learningcited by 212
  8. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  9. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  10. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  11. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 140
  12. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  13. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  14. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 115
  15. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  16. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 80
  17. 2025A Survey on Diffusion Language Modelscited by 75
  18. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Modelscited by 67
  19. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  20. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 48
  21. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 45
  22. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
  23. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  24. 2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 30
  25. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 26
  26. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 24
  27. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 20
  28. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  29. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  30. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 11
  31. 2025Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedulescited by 10
  32. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Modelscited by 8
  33. 2025Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibrationcited by 7
  34. 2026DLLM-Searcher: Adapting Diffusion Large Language Model for Search Agentscited by 7
  35. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  36. 2026DLLM Agent: See Farther, Run Fastercited by 5
  37. 2026Efficient and Stable Reinforcement Learning for Diffusion Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Modelscited by 3
  2. 2026Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Modelscited by 0