mlsanityData to 7 Oct 2026

FastGuide: Accelerating Reward Guidance for Diffusion Large Language Models

arXiv 2609.36202 ↗First posted 28 Sep 2026cs.CL cs.LGcited by 0

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Gradient-based reward guidance provides a flexible way to use downstream reward models to control masked diffusion language models at inference time. However, its computational cost remains high as each decoding iteration incurs expensive diffusion model forward passes and reward model backpropagation steps. To address this, we introduce FastGuide, an adaptive hybrid of parallel and autoregressive decoding to accelerate reward guidance for diffusion language models. In analogy to parallel decoding, FastGuide amortizes the cost of reward model backpropagation by computing guidance once per decoding step and reusing it to generate multiple tokens. Within each decoding step, FastGuide makes diffusion forward passes autoregressive by unmasking tokens one at a time while efficiently recomputing token distributions after each unmasking by utilizing KV caching techniques and sparse recomputation of attention. Lastly, to adapt hybrid decoding to the model's confidence, FastGuide defers any token that the model is unconfident about under its recomputed distribution. Experiments on three reward benchmarks demonstrate that FastGuide is up to 4.4× faster than sequential reward-guided decoding while retaining similar generation quality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 56 of its 57 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

56 of 57 references.

Closest ancestors · 16

Same direction, newest first

  1. 2026SimSD: Simple Speculative Decoding in Diffusion Language Models1 Jun
  2. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation26 Mar
  3. 2026DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention9 Mar
  4. 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding5 Feb
  5. 2026Entropy Aware Reward Guidance for Diffusion Language Model Alignment4 Feb
  6. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
  7. 2025Test-Time Anchoring for Discrete Diffusion Posterior Sampling2 Oct
  8. 2025Dream 7B: Diffusion Large Language Models21 Aug
  9. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  10. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  11. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  12. 2025Large Language Diffusion Models14 Feb
4 more
  1. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  2. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
  3. 2024Discrete Copula Diffusion2 Oct
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun

Other references · 40

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  5. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  6. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  7. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  8. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  9. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  10. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 218
  11. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  12. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 201
28 more
  1. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  2. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  3. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
  4. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  5. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
  6. 2023Controlled Decoding from Language Modelscited by 133
  7. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 115
  8. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  9. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoningcited by 106
  10. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 85
  11. 2025Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Expertscited by 85
  12. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 45
  13. 2024Cascade Reward Sampling for Efficient Decoding-Time Alignmentcited by 38
  14. 2024LMUnit: Fine-grained Evaluation with Natural Language Unit Testscited by 32
  15. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 32
  16. 2023Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Modelscited by 29
  17. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 29
  18. 2025Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphscited by 21
  19. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  20. 2025Unveiling the Potential of Diffusion Large Language Model in Controllable Generationcited by 14
  21. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Modelscited by 13
  22. 2026DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMscited by 13
  23. 2025LazyMAR: Accelerating Masked Autoregressive Models via Feature Cachingcited by 11
  24. 2024Frequency-Guided Posterior Sampling for Diffusion-Based Image Restorationcited by 10
  25. 2024Compress Guidance in Conditional Diffusion Samplingcited by 4
  26. 2024G2D2: Gradient-Guided Discrete Diffusion for Inverse Problem Solvingcited by 4
  27. 2026FOCUS: DLLMs Know How to Tame Their Compute Boundcited by 3
  28. 2026Lookahead Sample Reward Guidance for Test-Time Scaling of Diffusion Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs
  2. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  3. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
  4. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
  5. 2026Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models
  6. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding
  7. 2025dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learning
  8. 2025MRO: Enhancing Reasoning in Diffusion Language Models via Multi-Reward Optimization