mlsanityData to 7 Oct 2026

Efficient and Stable Reinforcement Learning for Diffusion Language Models

arXiv 2602.08905 ↗First posted 9 Feb 2026cs.AIcited by 2

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

Reinforcement Learning (RL) is crucial for unlocking the complex reasoning capabilities of Diffusion-based Large Language Models (dLLMs). However, applying RL to dLLMs faces unique challenges in efficiency and stability. To address these challenges, we propose Spatio-Temporal Pruning (STP), a framework designed to simultaneously improve the efficiency and stability of RL for dLLMs. STP compresses the redundancy in the generative process through: (1) textitspatial pruning, which constrains the exploration space using static priors; and (2) textittemporal pruning, which bypasses redundant late-stage refinement steps. Our theoretical analysis demonstrates that STP strictly reduces the variance of the log-likelihood estimation, thereby ensuring more stable policy updates. Extensive experiments demonstrate that STP surpasses state-of-the-art baselines in both efficiency and accuracy. Our code is available at https://github.com/Lolo1222/STP.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 24 of its 26 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

24 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models7 Jul
  2. 2025Mercury: Ultra-Fast Language Models Based on Diffusion17 Jun
  3. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  4. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning16 Apr

Other references · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  8. 2025Large Language Diffusion Modelscited by 872
  9. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  10. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  11. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  12. 2025Dream 7B: Diffusion Large Language Modelscited by 515
8 more
  1. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  2. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  3. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  4. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  5. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  6. 2024Reinforcement Learning for LLM Post-Training: A Surveycited by 91
  7. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 38
  8. 2023Semi-Offline Reinforcement Learning for Optimized Text Generationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLMcited by 2
  2. 2026SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
  2. 2025d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models
  3. 2026dTRPO: Trajectory Reduction in Policy Optimization of Diffusion Large Language Models
  4. 2026DARE: Diffusion Large Language Models Alignment and Reinforcement Executor
  5. 2025Inpainting-Guided Policy Optimization for Diffusion Large Language Models
  6. 2025Reasoning in Diffusion Large Language Models is Concentrated in Dynamic Confusion Zones
  7. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
  8. 2026A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions