mlsanityData to 14 Oct 2026

DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Discrete Diffusion Models

arXiv 2604.24357 ↗First posted 27 Apr 2026cs.LG cs.AIcited by 3

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Discrete diffusion models admit many token orders, yet most systems rely on confidence-based decoding. Confidence is a strong and efficient heuristic, but it can be myopic because local certainty does not measure a position's effect on terminal quality. We introduce DPRM (Doob h-transform Process Reward Model), a plug-in token-ordering module induced by a terminal-reward-tilted trajectory distribution. The Doob transform converts terminal reward into a process reward for each candidate position. A compact estimator indexed by generation progress, confidence, and optional positional or task state makes this correction reusable across partial states. DPRM changes only token order; the host architecture, denoising objective, supervision, and token sampler remain fixed. We derive the exact reward-tilted law and bound its shortlist and online approximations. Across nine recent open-source hosts spanning language, multimodal generation, and scientific sequences, DPRM improves reasoning, numeric VQA, visual-codebook ordering, and preference-conditioned generation. Relative to confidence, it improves PUMA GSM8K by 14.6%, Omni-Diffusion CLIP-L/14 by 13.2%, Prism voted accuracy from 82.41% to 83.85%, and RealWorldQA numeric/count accuracy by 8.97 points. Matched scientific gains include 8.3% in DCM nonzero recovery, 15.0% in GenMol QED, and 53.3% in SDPO-DNA total utility. Entropy and random-order controls show that the gains do not come from favoring uncertain tokens indiscriminately. Token-order traces and shared-state interventions instead show how terminal reward can correct a locally confident but globally poor choice. Code is available at: https://github.com/DakeBU/DPRM-DLLM.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 44 of its 57 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

44 of 57 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Models23 Mar
  2. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training10 Feb
  3. 2026Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models2 Feb
  4. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models21 Jan
  5. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  6. 2025Learning Unmasking Policies for Diffusion Language Models9 Dec
  7. 2025Planned Diffusion20 Oct
  8. 2025Dream 7B: Diffusion Large Language Models21 Aug
  9. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  10. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  11. 2025Large Language Diffusion Models14 Feb
  12. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
1 more
  1. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct

Other references · 31

  1. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  2. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  3. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  4. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  5. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  6. 2013A survey of the Schr"odinger problem and some of its connections with optimal transportcited by 292
  7. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  8. 2019SGD: General Analysis and Improved Ratescited by 207
  9. 2013Non-stationary Stochastic Optimizationcited by 175
  10. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  11. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  12. 2024Diffusion Language Models Are Versatile Protein Learnerscited by 98
19 more
  1. 2014Stochastic Optimization with Importance Samplingcited by 91
  2. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 79
  3. 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 64
  4. 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 54
  5. 2025GenMol: A Drug Discovery Generalist with Discrete Diffusioncited by 50
  6. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 46
  7. 2025Soft Best-of-n Sampling for Model Alignmentcited by 30
  8. 2025Learning-Order Autoregressive Models with Application to Molecular Graph Generationcited by 29
  9. 2024Self-Guidance: Boosting Flow and Diffusion Generation on Their Owncited by 23
  10. 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
  11. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 20
  12. 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusioncited by 13
  13. 2024PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inferencecited by 10
  14. 2021Stochastic Reweighted Gradient Descentcited by 7
  15. 2026D3LM: A Discrete DNA Diffusion Language Model for Bidirectional DNA Understanding and Generationcited by 6
  16. 2026Lingshu-Cell: A generative cellular world model for transcriptome modeling toward virtual cellscited by 6
  17. 2024Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samplescited by 5
  18. 2025Distributional Biases in Post-Training: A Markovian Analysis of Reasoning Trajectoriescited by 4
  19. 2025Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Trainingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Grab a Coffee: Future-Aware Guidance for Discrete Diffusion with Compiled Objectivescited by 0
  2. 2026D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transformcited by 0
  3. 2026Bayesian Entropy-based Reordering for Calibrated Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
  2. 2025Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs
  3. 2026NI Sampling: Accelerating Discrete Diffusion Sampling by Token Order Optimization
  4. 2025Path Planning for Masked Diffusion Model Sampling
  5. 2026Adaptive Order Policies for Masked Diffusion
  6. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Models
  7. 2026Latent Reward Registers for Diffusion Preference Alignment
  8. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models