# Distribution Matching Distillation Meets Reinforcement Learning

arXiv: https://arxiv.org/abs/2511.13649 · HTML: https://mlsanity.com/paper/2511.13649/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 79 of its 91 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)
- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 30 of 79)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)

## Built on by (most cited first, 30 of 50)

- [Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer](https://mlsanity.com/paper/2511.22699/) (2025)
- [Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models](https://mlsanity.com/paper/2606.25473/) (2026)
- [D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models](https://mlsanity.com/paper/2605.05204/) (2026)
- [TwinFlow: Realizing One-step Generation on Large Models with Self-adversarial Flows](https://mlsanity.com/paper/2512.05150/) (2025)
- [Manifold-Aware Exploration for Reinforcement Learning in Video Generation](https://mlsanity.com/paper/2603.21872/) (2026)
- [DNA: Uncovering Universal Latent Forgery Knowledge](https://mlsanity.com/paper/2601.22515/) (2026)
- [Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation](https://mlsanity.com/paper/2604.23632/) (2026)
- [Reinforcing Few-step Generators via Reward-Tilted Distribution Matching](https://mlsanity.com/paper/2605.26108/) (2026)
- [Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models](https://mlsanity.com/paper/2605.21573/) (2026)
- [ERNIE-Image Technical Report](https://mlsanity.com/paper/2605.25347/) (2026)
- [An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models](https://mlsanity.com/paper/2608.16887/) (2026)
- [AdaTSQ: Pushing the Pareto Frontier of Diffusion Transformers via Temporal-Sensitivity Quantization](https://mlsanity.com/paper/2602.09883/) (2026)
- [MeanFlowNFT: Bringing Forward-Process RL to Average-Velocity Generators](https://mlsanity.com/paper/2607.15273/) (2026)
- [1.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillation](https://mlsanity.com/paper/2604.04018/) (2026)
- [Continuous-Time Distribution Matching for Few-Step Diffusion Distillation](https://mlsanity.com/paper/2605.06376/) (2026)
- [Duality Models: An Embarrassingly Simple One-step Generation Paradigm](https://mlsanity.com/paper/2602.17682/) (2026)
- [Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE](https://mlsanity.com/paper/2605.02641/) (2026)
- [CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2605.25378/) (2026)
- [CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation](https://mlsanity.com/paper/2606.21982/) (2026)
- [Advances in GRPO for Generation Models: A Survey](https://mlsanity.com/paper/2603.06623/) (2026)
- [HybridStitch: Pixel and Timestep Level Model Stitching for Diffusion Acceleration](https://mlsanity.com/paper/2603.07815/) (2026)
- [Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learning](https://mlsanity.com/paper/2604.19009/) (2026)
- [Stream-R1: Reliability-Perplexity Aware Reward Distillation for Streaming Video Generation](https://mlsanity.com/paper/2605.03849/) (2026)
- [Delta Forcing: Trust Region Steering for Interactive Autoregressive Video Generation](https://mlsanity.com/paper/2605.14382/) (2026)
- [High-Fidelity Two-Step Image Generation via Teacher-Aligned End-to-End Distillation](https://mlsanity.com/paper/2606.12575/) (2026)
- [Cross-Resolution Distribution Matching for Diffusion Distillation](https://mlsanity.com/paper/2603.06136/) (2026)
- [Rdm: Re-conceptualizing Distribution Matching as a Reward for Diffusion Distillation](https://mlsanity.com/paper/2603.28460/) (2026)
- [AdvDMD: Adversarial Reward Meets DMD For High-Quality Few-Step Generation](https://mlsanity.com/paper/2604.28126/) (2026)
- [FlashMol: High-Quality Molecule Generation in as Few as Four Steps](https://mlsanity.com/paper/2605.07020/) (2026)
- [Efficient Image Synthesis with Sphere Latent Encoder](https://mlsanity.com/paper/2605.15592/) (2026)

## Abstract (arXiv)

Distribution Matching Distillation (DMD) facilitates efficient inference by distilling multi-step diffusion models into few-step variants. Concurrently, Reinforcement Learning (RL) has emerged as a vital tool for aligning generative models with human preferences. While both represent critical post-training stages for large-scale diffusion models, existing studies typically treat them as independent, sequential processes, leaving a systematic framework for their unification largely unexplored. In this work, we demonstrate that jointly optimizing these two objectives yields mutual benefits: RL enables more preference-aware and controllable distillation rather than uniformly compressing the full data distribution, while DMD serves as an effective regularizer to mitigate reward hacking during RL training. Building on these insights, we propose DMDR, a unified framework that incorporates Reward-Tilted Distribution Matching optimization alongside two dynamic distillation training strategies in the initial stage, followed by the joint DMD and RL optimization in the second stage. Extensive experiments demonstrate that DMDR achieves state-of-the-art visual quality and prompt adherence among few-step generation methods, even surpassing the performance of its multi-step teacher model.