mlsanityData to 14 Oct 2026

Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models

arXiv 2505.10446 ↗First posted 15 May 2025cs.CLcited by 54

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

We introduce the Diffusion Chain of Lateral Thought (DCoLT), a reasoning framework for diffusion language models. DCoLT treats each intermediate step in the reverse diffusion process as a latent "thinking" action and optimizes the entire reasoning trajectory to maximize the reward on the correctness of the final answer with outcome-based Reinforcement Learning (RL). Unlike traditional Chain-of-Thought (CoT) methods that follow a causal, linear thinking process, DCoLT allows bidirectional, non-linear reasoning with no strict rule on grammatical correctness amid its intermediate steps of thought. We implement DCoLT on two representative Diffusion Language Models (DLMs). First, we choose SEDD as a representative continuous-time discrete diffusion model, where its concrete score derives a probabilistic policy to maximize the RL reward over the entire sequence of intermediate diffusion steps. We further consider the discrete-time masked diffusion language model -- LLaDA, and find that the order to predict and unmask tokens plays an essential role to optimize its RL action resulting from the ranking-based Unmasking Policy Module (UPM) defined by the Plackett-Luce model. Experiments on both math and code generation tasks show that using only public data and 16 H800 GPUs, DCoLT-reinforced DLMs outperform other DLMs trained by SFT or RL or even both. Notably, DCoLT-reinforced LLaDA boosts its reasoning accuracy by +9.8%, +5.7%, +11.4%, +19.5% on GSM8K, MATH, MBPP, and HumanEval.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 29 of its 49 references that are in the map. First posted on arXiv 15 May 2025.

Builds on

29 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning16 Apr

Other references · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  10. 2025s1: Simple test-time scalingcited by 1,341
  11. 2024OpenAI o1 System Cardcited by 1,244
  12. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
16 more
  1. 2025Large Language Diffusion Modelscited by 912
  2. 2024DeepSeek-V3 Technical Reportcited by 811
  3. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  4. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  5. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  6. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  7. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  8. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  9. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 208
  10. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 176
  11. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  12. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 119
  13. 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 108
  14. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methodscited by 38
  15. 2024Schedule On the Fly: Diffusion Time Prediction for Faster and Better Image Generationcited by 30
  16. 2018Choosing to Rankcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 29.

54 papers.

Sort:

By area:

  1. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  3. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  4. 2025A Survey on Latent Reasoningcited by 81
  5. 2025A Survey on Diffusion Language Modelscited by 78
  6. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Modelscited by 71
  7. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  8. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 57
  9. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMscited by 40
  10. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
  11. 2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 34
  12. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 33
38 more
  1. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 30
  2. 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policiescited by 22
  3. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 19
  4. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Stepcited by 17
  5. 2025d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Modelscited by 17
  6. 2025dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learningcited by 10
  7. 2025From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Modelcited by 9
  8. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Modelscited by 8
  9. 2026LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learningcited by 7
  10. 2025Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertaintycited by 6
  11. 2025Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Modelcited by 6
  12. 2026Unifying Masked Diffusion Models with Various Generation Orders and Beyondcited by 6
  13. 2026GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Modelscited by 6
  14. 2025D3ToM: Decider-Guided Dynamic Token Merging for Accelerating Diffusion MLLMscited by 5
  15. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Modelscited by 5
  16. 2026Learning from the Self-future: On-policy Self-distillation for dLLMscited by 5
  17. 2025GIFT: Guided Importance-Aware Fine-Tuning for Diffusion Language Modelscited by 4
  18. 2025Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Modelscited by 4
  19. 2025Reasoning in Diffusion Large Language Models is Concentrated in Dynamic Confusion Zonescited by 3
  20. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Modelscited by 3
  21. 2026MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Modelscited by 3
  22. 2026dTRPO: Trajectory Reduction in Policy Optimization of Diffusion Large Language Modelscited by 3
  23. 2026Mask-Aware Policy Gradients for Diffusion Language Modelscited by 3
  24. 2026Efficient and Stable Reinforcement Learning for Diffusion Language Modelscited by 2
  25. 2026dFlowGRPO: Rate-Aware Policy Optimization for Discrete Flow Modelscited by 2
  26. 2026A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Modelscited by 2
  27. 2026Lookahead Path Likelihood Optimization for Diffusion LLMscited by 1
  28. 2026DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inferencecited by 1
  29. 2026The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Modelscited by 1
  30. 2026MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Modelscited by 1
  31. 2026Hesitation-Aware On-Policy Distillation for Diffusion Language Modelscited by 1
  32. 2026FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generationcited by 0
  33. 2026AR-MAP: Are Autoregressive Large Language Models Implicit Teachers for Diffusion Large Language Models?cited by 0
  34. 2026MemDLM: Memory-Enhanced DLM Trainingcited by 0
  35. 2026Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Modelscited by 0
  36. 2026Reinforcement Learning from Denoising Feedbackcited by 0
  37. 2026Efficient Reinforcement for Visual-Textual Thinking with Discrete Diffusion Modelcited by 0
  38. 2026SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicingcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning
  2. 2025Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards
  3. 2024Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models
  4. 2026Reasoning with Latent Tokens in Diffusion Language Models
  5. 2025Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning
  6. 2026Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space
  7. 2025CTRLS: Chain-of-Thought Reasoning via Latent State-Transition
  8. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models