mlsanityData to 7 Oct 2026

Think While You Generate: Discrete Diffusion with Planned Denoising

arXiv 2410.06264 ↗First posted 8 Oct 2024cs.LG cs.AI cs.CLcited by 61

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

Discrete diffusion has achieved state-of-the-art performance, outperforming or approaching autoregressive models on standard benchmarks. In this work, we introduce Discrete Diffusion with Planned Denoising (DDPD), a novel framework that separates the generation process into two models: a planner and a denoiser. At inference time, the planner selects which positions to denoise next by identifying the most corrupted positions in need of denoising, including both initially corrupted and those requiring additional refinement. This plan-and-denoise approach enables more efficient reconstruction during generation by iteratively identifying and denoising corruptions in the optimal order. DDPD outperforms traditional denoiser-only mask diffusion methods, achieving superior results on language modeling benchmarks such as text8, OpenWebText, and token-based image generation on ImageNet 256 × 256. Notably, in language modeling, DDPD significantly reduces the performance gap between diffusion-based and autoregressive methods in terms of generative perplexity. Code is available at https://github.com/liusulin/DDPD.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 7 of its 47 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

7 of 47 references.

References · 7

  1. 2017Decoupled Weight Decay Regularizationcited by 1,722
  2. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  4. 2024Discrete Flow Matchingcited by 301
  5. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  6. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  7. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Timecited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 35; 2026: 25.

61 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 208
  3. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  4. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  5. 2025A Survey on Diffusion Language Modelscited by 75
  6. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  7. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  8. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  9. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Modelscited by 34
  10. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  11. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 30
  12. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
38 more
  1. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  2. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  3. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 20
  4. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  5. 2025Unifying Autoregressive and Diffusion-Based Sequence Generationcited by 17
  6. 2025Anchored Diffusion Language Modelcited by 17
  7. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 15
  8. 2025Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decodingcited by 14
  9. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusioncited by 14
  10. 2025Planner Aware Path Learning in Diffusion Language Models Trainingcited by 14
  11. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testingcited by 13
  12. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximationscited by 13
  13. 2026The Diffusion Duality, Chapter II: Psi-Samplerscited by 12
  14. 2024[MASK] is All You Needcited by 11
  15. 2026Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulationcited by 11
  16. 2025Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generationcited by 9
  17. 2025Enabling Approximate Joint Sampling in Diffusion LMscited by 9
  18. 2025Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flowscited by 8
  19. 2025Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuningcited by 8
  20. 2025Planned Diffusioncited by 8
  21. 2025Enabling Autoregressive Models to Fill In Masked Tokenscited by 7
  22. 2025Steering Generative Models with Experimental Data for Protein Fitness Optimizationcited by 7
  23. 2025On the Reasoning Abilities of Masked Diffusion Language Modelscited by 6
  24. 2025Guided Star-Shaped Masked Diffusioncited by 5
  25. 2026IDLM: Inverse-distilled Diffusion Language Modelscited by 5
  26. 2026Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Modelscited by 4
  27. 2025Guided Transfer Learning for Discrete Diffusion Modelscited by 3
  28. 2026Reward-Guided Discrete Diffusion via Clean-Sample Markov Chain for Molecule and Biological Sequence Designcited by 3
  29. 2026Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learningcited by 3
  30. 2026Training-Free Self-Correction for Multimodal Masked Diffusion Modelscited by 2
  31. 2026Fixed-Point Masked Generative Modelingcited by 2
  32. 2026A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decodingcited by 2
  33. 2026AR-MAP: Are Autoregressive Large Language Models Implicit Teachers for Diffusion Large Language Models?cited by 1
  34. 2026CRoCoDiL: Continuous and Robust Conditioned Diffusion for Languagecited by 1
  35. 2026STDec: Spatio-Temporal Stability Guided Decoding for dLLMscited by 1
  36. 2026DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Discrete Diffusion Modelscited by 1
  37. 2026Accelerating Discrete Diffusion Models with Parallel-In-Time Samplingcited by 1
  38. 2025Latent-Augmented Discrete Diffusion Modelscited by 0

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Discrete Diffusion Models for Language Generation
  2. 2024Step Saver: Predicting Minimum Denoising Steps for Diffusion Model Image Generation
  3. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spaces
  5. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
  6. 2026Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
  7. 2025Compressed Image Generation with Denoising Diffusion Codebook Models
  8. 2023Denoising Diffusion Bridge Models