mlsanityData to 14 Oct 2026

Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models

arXiv 2605.11726 ↗First posted 12 May 2026cs.LGcited by 0

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

Abstract (arXiv)

Recently, reinforcement learning (RL) has been widely applied during post-training for diffusion large language models (dLLMs) to enhance reasoning with block-wise semi-autoregressive generation. Block size has therefore become a vital factor in dLLMs, since it determines the parallel decoding granularity and affects the rollout trajectories during RL optimisation, e.g., GRPO. Instead of investigating the effect of block size during inference on individual domains, this paper studies block size from a domain conflict perspective for dLLM RL post-training in multi-domain scenarios. The main contributions are: (1) a formulation of domain block size conflict in multi-domain RL for dLLMs, which will largely affect the post-training effectiveness for rollout-based RL methods; (2) a novel dataset, Block-R1-41K is constructed with a best-improved training block size for each sample, which also induces a Block Size Conflict Score to quantitatively measure the domain conflict; (3) a new benchmark, Block-R1, for flexible RL post-training for dLLMs in both single and cross domain; and (4) a simple yet powerful cross-domain post-training method with sample-level best-improved training block sizes. Extensive experiments on 13 distinct datasets, 7 latest RL algorithms and diverse dLLM backbones are comprehensively covered in Block-R1. The benchmark is open-sourced at https://github.com/YanJiangJerry/Block-R1 with the dataset released at https://huggingface.co/datasets/YanJiangJerry/Block-R1-41K.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 53 of its 79 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

53 of 79 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning4 May
  2. 2025Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Experts6 Oct
  3. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation25 Sep
  4. 2025Mercury: Ultra-Fast Language Models Based on Diffusion17 Jun
  5. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  6. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct

Other references · 47

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2017Decoupled Weight Decay Regularizationcited by 1,831
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  11. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  12. 2025Large Language Diffusion Modelscited by 912
35 more
  1. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  3. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  5. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  6. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  7. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  8. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  10. 2024Scaling up Masked Diffusion Models on Textcited by 199
  11. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  12. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
  13. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 162
  14. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforcecited by 145
  15. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 110
  16. 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplificationcited by 110
  17. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 79
  18. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learningcited by 73
  19. 2025Efficient Online Reinforcement Learning for Diffusion Policycited by 71
  20. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 54
  21. 2023Fine-Tuning Language Models with Advantage-Induced Policy Alignmentcited by 48
  22. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methodscited by 38
  23. 2025NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoningcited by 38
  24. 2025dInfer: An Efficient Inference Framework for Diffusion Language Modelscited by 37
  25. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  26. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 31
  27. 2023Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Modelscited by 17
  28. 2025CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generationcited by 17
  29. 2026Multiplex Thinking: Reasoning via Token-wise Branch-and-Mergecited by 16
  30. 2025RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignmentcited by 14
  31. 2026Deferred Commitment Decoding for Diffusion Language Modelscited by 12
  32. 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Modelscited by 10
  33. 2026Stabilizing Reinforcement Learning for Diffusion Language Modelscited by 10
  34. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  35. 2026DARE: Diffusion Large Language Models Alignment and Reinforcement Executorcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning
  2. 2025d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models
  3. 2025Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models
  4. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
  5. 2026A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
  6. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models
  7. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
  8. 2026How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws