DARE: Diffusion Large Language Models Alignment and Reinforcement Executor
On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
Abstract (arXiv)
Diffusion large language models (dLLMs) are emerging as a compelling alternative to dominant autoregressive models, replacing strictly sequential token generation with iterative denoising and parallel generation dynamics. However, their open-source ecosystem remains fragmented across model families and, in particular, across post-training pipelines, where reinforcement learning objectives, rollout implementations and evaluation scripts are often released as paper-specific codebases. This fragmentation slows research iteration, raises the engineering burden of reproduction, and makes fair comparison across algorithms difficult. We present DARE (dLLMs Alignment and Reinforcement Executor), an open framework for post-training and evaluating dLLMs. Built on top of verl~citesheng2024hybridflow and OpenCompass~cite2023opencompass, DARE unifies supervised fine-tuning, parameter-efficient fine-tuning, preference optimization, and dLLM-specific reinforcement learning under a shared execution stack for both masked and block diffusion language models. Across representative model families including LLaDA, Dream, SDAR, and LLaDA2.x, DARE provides broad algorithmic coverage, reproducible benchmark evaluation, and practical acceleration. Extensive empirical results position that DARE serves as a reusable research substrate for developing, comparing, and deploying post-training methods for current and emerging dLLMs.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 50 of its 57 references that are in the map. First posted on arXiv 5 Apr 2026.
Builds on
50 of 57 references.
References · 50
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023Let's Verify Step by Stepcited by 3,598
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
38 more
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
- 2025Large Language Diffusion Modelscited by 872
- 2024Simple and Effective Masked Diffusion Language Modelscited by 800
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2025Dream 7B: Diffusion Large Language Modelscited by 515
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
- 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
- 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
- 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
- 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
- 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
- 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
- 2024Scaling up Masked Diffusion Models on Textcited by 195
- 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
- 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
- 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
- 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
- 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
- 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
- 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
- 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 80
- 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 54
- 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 48
- 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 38
- 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 32
- 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
- 2025Unifying Autoregressive and Diffusion-Based Sequence Generationcited by 17
- 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 17
- 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Stepcited by 16
- 2025LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMindcited by 10
- 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Modelscited by 10
- 2025Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Modelscited by 9
- 2026ρ-textttEOS: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMscited by 7
- 2025Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Modelscited by 3
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026dOPSD: On-Policy Self-Distillation for Diffusion Language Modelscited by 4
- 2026From Table to Cell: Attention for Better Reasoning with TABALIGNcited by 1
- 2026CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Modelscited by 1
- 2026Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Modelscited by 0
- 2026When Top-1 Fails: Calibrating LoRA Monitors for Masked Diffusion LMscited by 0
- 2026dOPSD: On-Policy Self-Distillation for Diffusion Language Models5 Jul 2026
- 2026When Top-1 Fails: Calibrating LoRA Monitors for Masked Diffusion LMs23 Jun 2026
- 2026CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Models4 Jun 2026
- 2026From Table to Cell: Attention for Better Reasoning with TABALIGN14 May 2026
- 2026Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models12 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026DARE: Diffusion Language Model Activation Reuse for Efficient Inference
- 2025d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models
- 2025Learning Unmasking Policies for Diffusion Language Models
- 2026dTRPO: Trajectory Reduction in Policy Optimization of Diffusion Large Language Models
- 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
- 2026Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
- 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
- 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping