mlsanityData to 14 Oct 2026

DiRL: An Efficient Post-Training Framework for Diffusion Language Models

arXiv 2512.22234 ↗First posted 23 Dec 2025cs.LG cs.AIcited by 10

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models. While recent efforts have validated their pre-training potential and accelerated inference speeds, the post-training landscape for dLLMs remains underdeveloped. Existing methods suffer from computational inefficiency and objective mismatches between training and inference, severely limiting performance on complex reasoning tasks such as mathematics. To address this, we introduce DiRL, an efficient post-training framework that tightly integrates FlexAttention-accelerated blockwise training with LMDeploy-optimized inference. This architecture enables a streamlined online model update loop, facilitating efficient two-stage post-training (Supervised Fine-Tuning followed by Reinforcement Learning). Building on this framework, we propose DiPO, the first unbiased Group Relative Policy Optimization (GRPO) implementation tailored for dLLMs. We validate our approach by training DiRL-8B-Instruct on high-quality math data. Our model achieves state-of-the-art math performance among dLLMs and surpasses comparable models in the Qwen2.5 series on several benchmarks.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 41 of its 48 references that are in the map. First posted on arXiv 23 Dec 2025.

Builds on

41 of 48 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization9 Oct
  2. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  3. 2025Training Optimal Large Diffusion Language Models28 Sep
  4. 2025Inpainting-Guided Policy Optimization for Diffusion Large Language Models12 Sep
  5. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models8 Sep
  6. 2025Dream 7B: Diffusion Large Language Models21 Aug
  7. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  8. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  9. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs17 Jun
  10. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  11. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  12. 2025Large Language Diffusion Models14 Feb
2 more
  1. 2024Scaling up Masked Diffusion Models on Text24 Oct
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct

Other references · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  8. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  10. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  11. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  12. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
15 more
  1. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  2. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  3. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
  4. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  5. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  6. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  7. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 109
  8. 2024LongAlign: A Recipe for Long Context Alignment of Large Language Modelscited by 106
  9. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 70
  10. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 65
  11. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  12. 2025UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Modelscited by 13
  13. 2025Thus Spake Long-Context Large Language Modelcited by 12
  14. 2025VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Trainingcited by 9
  15. 2025Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrowcited by 11
  2. 2026DARE: Diffusion Large Language Models Alignment and Reinforcement Executorcited by 5
  3. 2026Adaptive Test-Time Compute Allocation for Block Diffusion Language Models in Complex Reasoningcited by 2
  4. 2026Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learningcited by 1
  5. 2026A Syllogistic Probe: Tracing the Evolution of Logic Reasoning in Large Language Modelscited by 0
  6. 2026Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Modelscited by 0
  7. 2026SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicingcited by 0
  8. 2026Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMscited by 0
  9. 2026d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Modelscited by 0
  10. 2026Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Simple Policy Gradients for Reasoning with Diffusion Language Models
  2. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models
  3. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
  4. 2026Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantages
  5. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models
  6. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
  7. 2026Learnability-Informed Fine-Tuning of Diffusion Language Models
  8. 2025d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models