mlsanityData to 7 Oct 2026

SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing

arXiv 2607.00208 ↗First posted 30 Jun 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning for diffusion large language models (dLLMs) has largely moved to trajectory-aware methods. The current state of the art, TraceRL, holds that random masking is mismatched with the model's inference trajectory, and it reconstructs that trajectory during training by slicing each rollout into up to K/s trajectory-aligned training samples, a cost that grows with the block size K. We show that this mismatch can be mitigated without reconstructing the trajectory. Our method, SLIM-RL, bounds the commit risk of each rollout step with a tau-budget decoder, reducing aggregate commit risk in the training data. During optimization, SLIM-RL trains on these risk-controlled rollouts with a trace-free random-masking objective that adapts variance-reduction tools, combining sequence-level importance sampling, deterministic quadrature over masking levels under a mean-preserving, monotonically decreasing per-block mask schedule that we introduce. On SDAR-4B, SLIM-RL matches TraceRL's best MATH500 accuracy on only 0.46x its training samples at block size 16, improving over TraceRL by 6.32% on MATH500 and 11.05% on GSM8K under matched dynamic sampling. At block size 4, the 4B SLIM-RL surpasses the larger LLaDA-8B and Dream-7B dLLMs on math, exceeding LLaDA-8B by 10.76% on MATH500 while staying below the autoregressive Qwen2.5-7B. On code, it improves over TraceRL by 4.20% on MBPP and 3.65% on HumanEval. The tau-budget decoder transfers training-free across LLaDA, Dream, and SDAR. The source code is available at https://github.com/laolaorkkkkk/SLIM-RL .

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 34 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

33 of 34 references.

References · 33

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  9. 2025Large Language Diffusion Modelscited by 872
  10. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  11. 2025Group Sequence Policy Optimizationcited by 788
  12. 2025Dream 7B: Diffusion Large Language Modelscited by 515
21 more
  1. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  2. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  3. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  4. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  5. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  6. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  7. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  8. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  9. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  10. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  11. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 80
  12. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Modelscited by 67
  13. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Modelscited by 53
  14. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  15. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 30
  16. 2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 30
  17. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimationcited by 27
  18. 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Modelscited by 10
  19. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  20. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculationcited by 5
  21. 2026Efficient and Stable Reinforcement Learning for Diffusion Language Modelscited by 2

Built on by

Not cited yet.