mlsanityData to 7 Oct 2026

Distribution Matching Distillation Meets Reinforcement Learning

arXiv 2511.13649 ↗First posted 17 Nov 2025cs.CVcited by 43

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Distribution Matching Distillation (DMD) facilitates efficient inference by distilling multi-step diffusion models into few-step variants. Concurrently, Reinforcement Learning (RL) has emerged as a vital tool for aligning generative models with human preferences. While both represent critical post-training stages for large-scale diffusion models, existing studies typically treat them as independent, sequential processes, leaving a systematic framework for their unification largely unexplored. In this work, we demonstrate that jointly optimizing these two objectives yields mutual benefits: RL enables more preference-aware and controllable distillation rather than uniformly compressing the full data distribution, while DMD serves as an effective regularizer to mitigate reward hacking during RL training. Building on these insights, we propose DMDR, a unified framework that incorporates Reward-Tilted Distribution Matching optimization alongside two dynamic distillation training strategies in the initial stage, followed by the joint DMD and RL optimization in the second stage. Extensive experiments demonstrate that DMDR achieves state-of-the-art visual quality and prompt adherence among few-step generation methods, even surpassing the performance of its multi-step teacher model.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 79 of its 91 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

79 of 91 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Process19 Sep
  2. 2025Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference8 Sep
  3. 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
  4. 2025Flow-GRPO: Training Flow Matching Models via Online RL8 May
  5. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences24 Oct
  6. 2024Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models1 May
  7. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  8. 2023Training Diffusion Models with Reinforcement Learning22 May
  9. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference25 Mar

Other references · 70

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Flow Matching for Generative Modelingcited by 5,426
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
38 more
  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  6. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  8. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  9. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  10. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  11. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  12. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  13. 2023Adversarial Diffusion Distillationcited by 741
  14. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
  15. 2018Which Training Methods for GANs do actually Converge?cited by 672
  16. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  17. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  18. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
  19. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  20. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  21. 2024One Step Diffusion via Shortcut Modelscited by 373
  22. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 344
  23. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  24. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  25. 2025Improving Video Generation with Human Feedbackcited by 267
  26. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  27. 2023Data Filtering Networkscited by 247
  28. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  29. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 232
  30. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
  31. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  32. 2024Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesiscited by 153
  33. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 148
  34. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 142
  35. 2017Stabilizing Training of Generative Adversarial Networks through Regularizationcited by 118
  36. 2025ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generationcited by 94
  37. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 93
  38. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 85

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 41.

43 papers.

Sort:

By area:

  1. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 247
  2. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 23
  3. 2025TwinFlow: Realizing One-step Generation on Large Models with Self-adversarial Flowscited by 17
  4. 2026D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Modelscited by 17
  5. 2026Manifold-Aware Exploration for Reinforcement Learning in Video Generationcited by 14
  6. 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
  7. 2026DNA: Uncovering Universal Latent Forgery Knowledgecited by 10
  8. 2026Reinforcing Few-step Generators via Reward-Tilted Distribution Matchingcited by 6
  9. 2026ERNIE-Image Technical Reportcited by 5
  10. 2026AdaTSQ: Pushing the Pareto Frontier of Diffusion Transformers via Temporal-Sensitivity Quantizationcited by 4
  11. 2026Continuous-Time Distribution Matching for Few-Step Diffusion Distillationcited by 4
  12. 2026Lens: Rethinking Training Efficiency for Foundational Text-to-Image Modelscited by 4
31 more
  1. 2026MeanFlowNFT: Bringing Forward-Process RL to Average-Velocity Generatorscited by 4
  2. 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoEcited by 3
  3. 2026CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillationcited by 3
  4. 2026CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generationcited by 3
  5. 2026An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Modelscited by 3
  6. 2026Duality Models: An Embarrassingly Simple One-step Generation Paradigmcited by 2
  7. 2026HybridStitch: Pixel and Timestep Level Model Stitching for Diffusion Accelerationcited by 2
  8. 2026Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learningcited by 2
  9. 2026Stream-R1: Reliability-Perplexity Aware Reward Distillation for Streaming Video Generationcited by 2
  10. 2026High-Fidelity Two-Step Image Generation via Teacher-Aligned End-to-End Distillationcited by 2
  11. 2026Cross-Resolution Distribution Matching for Diffusion Distillationcited by 1
  12. 2026Advances in GRPO for Generation Models: A Surveycited by 1
  13. 2026Rdm: Re-conceptualizing Distribution Matching as a Reward for Diffusion Distillationcited by 1
  14. 2026AdvDMD: Adversarial Reward Meets DMD For High-Quality Few-Step Generationcited by 1
  15. 2026FlashMol: High-Quality Molecule Generation in as Few as Four Stepscited by 1
  16. 2026Delta Forcing: Trust Region Steering for Interactive Autoregressive Video Generationcited by 1
  17. 2026Efficient Image Synthesis with Sphere Latent Encodercited by 1
  18. 2026Monocular Avatar Reconstruction via Cascaded Diffusion Priors and UV-Space Differentiable Shadingcited by 1
  19. 2026AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatarscited by 1
  20. 2026Joint Alignment and Distillation for Video Generation via Sample-Guided Distribution Matchingcited by 1
  21. 2026Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillationcited by 0
  22. 2026World Model Self-Distillation: Training World Models to Solve General Taskscited by 0
  23. 2026A Test-time Actor-Critic Approach to News Images Generationcited by 0
  24. 2026Reward Lightning: Fast Video Generation via Homologous Preference Distillationcited by 0
  25. 2026ScaleResfusion: Residual Rectified Flow based on Residual Vector Fieldcited by 0
  26. 2026RL-Native Distillation: Exploiting Scored Trajectories for Few-Step Image Generationcited by 0
  27. 2026DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generationcited by 0
  28. 2026DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video World Modelscited by 0
  29. 2026PDMD: Projected Distribution Matching Distillation for Video Diffusion Modelscited by 0
  30. 2026PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillationcited by 0
  31. 2026DMA2: Pixel-space Distribution Matching with Adversarial and Anchor Lossescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ReDiF: Reinforced Distillation for Few Step Diffusion
  2. 2026ParetoSlider: Diffusion Models Post-Training for Continuous Reward Control
  3. 2026On-Policy Self-Distillation in Diffusion Models
  4. 2026Diffusion Fine-tuning with Rewarded Moment Matching Distillation
  5. 2025Data-regularized Reinforcement Learning for Diffusion Models at Scale
  6. 2026Scaling Reinforcement Learning for Diffusion Models via Velocity Matching
  7. 2025Beyond Reward Margin: Rethinking and Resolving Likelihood Displacement in Diffusion Models via Video Generation
  8. 2026Consistent Distribution Matching for Data-Free Diffusion Distillation