mlsanityData to 14 Oct 2026

Reasoning in Diffusion Large Language Models is Concentrated in Dynamic Confusion Zones

arXiv 2511.15208 ↗First posted 19 Nov 2025cs.LGcited by 3

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

Abstract (arXiv)

Diffusion Large Language Models (dLLMs) are rapidly emerging alongside autoregressive models as a powerful paradigm for complex reasoning, with reinforcement learning increasingly used for downstream alignment. Existing trajectory-based RL methods uniformly allocate policy gradients across denoising steps, implicitly treating all steps as equally important. We challenge this assumption by analyzing trajectories with several step-level metrics: entropy-based uncertainty, Confidence-Margin (CM) uncertainty, and Rate of Entropy Change (RoEC). These reveal structured "zones of confusion": transient spikes in uncertainty and instability that strongly predict final success or failure, while most steps remain stable. We propose Adaptive Trajectory Policy Optimization (ATPO), a lightweight step-selection strategy that dynamically reallocates gradient updates to these high-leverage steps without changing the RL objective, rewards, or compute budget. Using a hybrid RoEC+CM rule, ATPO delivers substantial gains in reasoning accuracy and training stability across benchmarks, showing that exploiting trajectory dynamics is key to advancing dLLM RL.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 41 of its 43 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

41 of 43 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models10 Oct
  2. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization9 Oct
  3. 2025Simple Policy Gradients for Reasoning with Diffusion Language Models5 Oct
  4. 2025Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards2 Oct
  5. 2025RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance29 Sep
  6. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation25 Sep
  7. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models7 Jul
  8. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  9. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning16 Apr

Other references · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2025Large Language Diffusion Modelscited by 912
  6. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  7. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  8. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  10. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  11. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 178
  12. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
20 more
  1. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  2. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  3. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  4. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 88
  5. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 84
  6. 2025Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Modelscited by 67
  7. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 65
  8. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 57
  9. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 41
  10. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMscited by 40
  11. 2025dInfer: An Efficient Inference Framework for Diffusion Language Modelscited by 37
  12. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 34
  13. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  14. 2025Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Maskingcited by 21
  15. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Stepcited by 17
  16. 2025Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inferencecited by 17
  17. 2025Inpainting-Guided Policy Optimization for Diffusion Large Language Modelscited by 16
  18. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 13
  19. 2025UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Modelscited by 13
  20. 2025Consolidating Reinforcement Learning for Multimodal Discrete Diffusion Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026LogicDiff: Logic-Guided Denoising Improves Zero-Shot Reasoning in Masked Diffusion Language Modelscited by 1
  2. 2026Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMscited by 0
  3. 2026Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning
  2. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
  3. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models
  4. 2025EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients
  5. 2026Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
  6. 2026Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models
  7. 2026dTRPO: Trajectory Reduction in Policy Optimization of Diffusion Large Language Models
  8. 2026Read the Trace, Steer the Path: Trajectory-Aware Reinforcement Learning for Diffusion Language Models