mlsanityData to 14 Oct 2026

Inference-Time Scaling of Diffusion Language Models via Trajectory Refinement

arXiv 2507.08390 ↗First posted 11 Jul 2025cs.LGcited by 7

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

Discrete diffusion models have recently emerged as strong alternatives to autoregressive language models, matching their performance through large-scale training. However, inference-time control remains relatively underexplored. In this work, we study how to steer generation toward desired rewards without retraining the models. Prior methods typically resample or filter within a single denoising trajectory, optimizing rewards step-by-step without trajectory-level refinement. We introduce particle Gibbs sampling for diffusion language models (PG-DLM), an inference-time algorithm enabling trajectory-level refinement. PG-DLM constructs a Markov chain over full denoising trajectories and applies a conditional sequential Monte Carlo kernel to resample them. By doing so, PG-DLM introduces a new scaling axis, the number of refinement iterations, which is unavailable to prior methods. Increasing iterations remains effective even as gains from adding more parallel samples saturate. Furthermore, PG-DLM enables adaptive compute allocation by performing additional iterations only when needed, leading to further efficiency gains. We derive theoretical guarantees for convergence and variance bounds, and analyze trade-offs across different scaling axes. Empirically, PG-DLM outperforms prior methods across compute budgets on reward-guided generation tasks. On GSM8K, it achieves 90.07% accuracy with 2.9 particles on average and 94.47% accuracy with 24.8 particles on average.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 49 of its 65 references that are in the map. First posted on arXiv 11 Jul 2025.

Builds on

49 of 65 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Inference-time Scaling of Diffusion Models through Classical Search29 May
  2. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning16 Apr
  3. 2025Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing25 Mar
  4. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  5. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps16 Jan
  6. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Models12 Jan

Other references · 43

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  11. 2018Neural Network Acceptability Judgmentscited by 958
  12. 2025Large Language Diffusion Modelscited by 912
31 more
  1. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  2. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  3. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  4. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  6. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  7. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
  8. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  9. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  10. 2024Discrete Flow Matchingcited by 317
  11. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 232
  12. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 208
  13. 2024Scaling up Masked Diffusion Models on Textcited by 199
  14. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  15. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  16. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 143
  17. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 136
  18. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  19. 2013Generalized Denoising Auto-Encoders as Generative Modelscited by 113
  20. 2016Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-controlcited by 101
  21. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  22. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 80
  23. 2019Elements of Sequential Monte Carlocited by 64
  24. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  25. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 46
  26. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 34
  27. 2024Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Modelscited by 25
  28. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 24
  29. 2024Diffusive Gibbs Samplingcited by 21
  30. 2023Moment Matching Denoising Gibbs Samplingcited by 7
  31. 2024Particle-Filtering-based Latent Diffusion for Inverse Problemscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2025RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidancecited by 7
  2. 2026ETS: Energy-Guided Test-Time Scaling for Training-Free RL Alignmentcited by 5
  3. 2026UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branchingcited by 2
  4. 2026The Safety-Aware Denoiser for Text Diffusion Modelscited by 2
  5. 2026VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusioncited by 1
  6. 2026Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoningcited by 0
  7. 2026Self-Repulsive Sampling for Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models
  2. 2025Navigating the Exploration-Exploitation Tradeoff in Inference-Time Scaling of Diffusion Models
  3. 2026S3: Stratified Scaling Search for Test-Time in Diffusion Language Models
  4. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  5. 2026Scaling Beyond Masked Diffusion Language Models
  6. 2026Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models
  7. 2026Consistent Diffusion Language Models
  8. 2026On Trajectory-Aware Training for Masked Diffusion Language Models