mlsanityData to 7 Oct 2026

Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement

arXiv 2511.05562 ↗First posted 4 Nov 2025cs.LG cs.AIcited by 7

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

Test-time scaling through reward-guided generation remains largely unexplored for discrete diffusion models despite its potential as a promising alternative. In this work, we introduce Iterative Reward-Guided Refinement (IterRef), a novel test-time scaling method tailored to discrete diffusion that leverages reward-guided noising-denoising transitions to progressively refine misaligned intermediate states. We formalize this process within a Multiple-Try Metropolis (MTM) framework, proving convergence to the reward-aligned distribution. Unlike prior methods that assume the current state is already aligned with the reward distribution and only guide the subsequent transition, our approach explicitly refines each state in situ, progressively steering it toward the optimal intermediate distribution. Across both text and image domains, we evaluate IterRef on diverse discrete diffusion models and observe consistent improvements in reward-guided generation quality. In particular, IterRef achieves striking gains under low compute budgets, far surpassing prior state-of-the-art baselines.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 37 of its 50 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

37 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Inference-time Scaling of Diffusion Models through Classical Search29 May
  2. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  3. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps16 Jan
  4. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Models12 Jan

Other references · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  11. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  12. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
21 more
  1. 2018Neural Network Acceptability Judgmentscited by 952
  2. 2025Large Language Diffusion Modelscited by 872
  3. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  4. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  5. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  6. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  7. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  8. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  9. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  10. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 218
  11. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  12. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  13. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Samplingcited by 124
  14. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 82
  15. 2023AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Accelerationcited by 70
  16. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimizationcited by 70
  17. 2023Guided Image Synthesis via Initial Image Editing in Diffusion Modelcited by 63
  18. 2023Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Latercited by 52
  19. 2024A Noise is Worth Diffusion Guidancecited by 48
  20. 2024Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Modelscited by 24
  21. 2025Position: Editing Large Language Models Poses Serious Safety Riskscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 48
  2. 2025Corrective Diffusion Language Modelscited by 11
  3. 2026VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editingcited by 1
  4. 2026Looped Diffusion Language Modelscited by 0
  5. 2026VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusioncited by 0
  6. 2026Spectral Feedback for Test-Time Alignment of Protein Diffusion Modelscited by 0
  7. 2026Does Uniform Discrete Diffusion Need Time?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Test-time Alignment of Diffusion Models without Reward Over-optimization
  2. 2026IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
  3. 2025Test-time scaling of diffusions with flow maps
  4. 2026Guided Trajectory Optimization with Sparse Scaling for Test-Time Diffusion
  5. 2026D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform
  6. 2026Hierarchical Variational Policies for Reward-Guided Diffusion
  7. 2025Rethinking Test Time Scaling for Flow-Matching Generative Models
  8. 2026Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement