# Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis

arXiv: https://arxiv.org/abs/2602.03139 · HTML: https://mlsanity.com/paper/2602.03139/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 53 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [Flow-GRPO: Training Flow Matching Models via Online RL](https://mlsanity.com/paper/2505.05470/) (2025)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022)
- [Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models](https://mlsanity.com/paper/2410.11081/) (2024)

## Built on by (most cited first, 20 of 20)

- [Qwen-Image-2.0 Technical Report](https://mlsanity.com/paper/2605.10730/) (2026)
- [Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models](https://mlsanity.com/paper/2606.25473/) (2026)
- [Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation](https://mlsanity.com/paper/2606.18478/) (2026)
- [Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation](https://mlsanity.com/paper/2604.10103/) (2026)
- [HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation](https://mlsanity.com/paper/2605.11596/) (2026)
- [STRIDE: Training-Free Diversity Guidance via PCA-Directed Feature Perturbation in Single-Step Diffusion Models](https://mlsanity.com/paper/2605.11494/) (2026)
- [CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation](https://mlsanity.com/paper/2606.21982/) (2026)
- [Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout](https://mlsanity.com/paper/2609.09123/) (2026)
- [Cross-Resolution Distribution Matching for Diffusion Distillation](https://mlsanity.com/paper/2603.06136/) (2026)
- [Manifold-Constrained Noise Optimization for Diverse Diffusion Sampling](https://mlsanity.com/paper/2607.23937/) (2026)
- [Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars](https://mlsanity.com/paper/2608.12107/) (2026)
- [Where and When to Force: Routed Forcing for Streaming Avatars](https://mlsanity.com/paper/2609.30963/) (2026)
- [Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation](https://mlsanity.com/paper/2606.03746/) (2026)
- [DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generation](https://mlsanity.com/paper/2608.09637/) (2026)
- [Uncertainty DMD: Restoring Diversity in Few-Step Autoregressive Video Distillation](https://mlsanity.com/paper/2609.11265/) (2026)
- [PDMD: Projected Distribution Matching Distillation for Video Diffusion Models](https://mlsanity.com/paper/2609.35768/) (2026)
- [Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation](https://mlsanity.com/paper/2609.37925/) (2026)
- [HelixWorld: A Real-time Interactive Audio-Visual World Model](https://mlsanity.com/paper/2609.38123/) (2026)
- [DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation](https://mlsanity.com/paper/2610.02188/) (2026)
- [Multi-Bandwidth Distribution Matching Distillation: On the Equivalence of Distribution Matching Distillation and Drifting Models](https://mlsanity.com/paper/2610.10989/) (2026)

## Abstract (arXiv)

Distribution matching distillation (DMD) facilitates few-step image generation by aligning a distilled student with a reference multi-step teacher. In practice, however, optimizing DMD can reduce sample diversity in few-step synthesis, and existing remedies typically rely on perceptual or adversarial regularization, leading to stability and scalability challenges during training. Here, we describe diversity-preserved DMD (DP-DMD), a role-separated distillation method inspired by the complementary roles of early and late denoising steps. Specifically, the first distillation step is trained with a teacher-derived target-prediction objective (e.g., v-prediction) to preserve sample diversity, while the remaining steps are optimized with the standard DMD loss to refine perceptual quality. DP-DMD, with no perceptual or adversarial regularization, no additional modules, and no teacher-generated reference samples, preserves sample diversity while maintaining competitive visual quality under few-step sampling, providing a simple and stable alternative to other DMD variants.