mlsanityData to 14 Oct 2026

Mean-Shift Distillation for Diffusion Mode Seeking

arXiv 2502.15989 ↗First posted 21 Feb 2025cs.LG cs.GRcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present mean-shift distillation, a novel diffusion distillation technique that provides a provably good proxy for the gradient of the diffusion output distribution. This is derived directly from mean-shift mode seeking on the distribution, and we show that its extrema are aligned with the modes. We further derive an efficient product distribution sampling procedure to evaluate the gradient. Our method is formulated as a drop-in replacement for score distillation sampling (SDS), requiring neither model retraining nor extensive modification of the sampling procedure. We show that it exhibits superior mode alignment as well as improved convergence in both synthetic and practical setups, yielding higher-fidelity results when applied to both text-to-image and text-to-3D applications with Stable Diffusion.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 32 of its 49 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

32 of 49 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024Classifier-Free Guidance is a Predictor-Corrector16 Aug
  2. 2024Guiding a Diffusion Model with a Bad Version of Itself4 Jun
  3. 2024EM Distillation for One-step Diffusion Models27 May
  4. 2024NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generation24 May
  5. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models11 Apr
  6. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  7. 2023A Geometric Perspective on Diffusion Models31 May
  8. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  9. 2022Classifier-Free Diffusion Guidance26 Jul
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
1 more
  1. 2020Denoising Diffusion Implicit Models6 Oct

Other references · 19

  1. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  4. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  5. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  6. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  7. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  8. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
  9. 2023Text-to-3D with Classifier Score Distillationcited by 128
  10. 2023Delta Denoising Scorecited by 121
  11. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  12. 2023Noise-Free Score Distillationcited by 99
7 more
  1. 2021Moser Flow: Divergence-based Generative Modeling on Manifoldscited by 67
  2. 2024Score Distillation via Reparametrized DDIMcited by 41
  3. 2023Taming Mode Collapse in Score Distillation for Text-to-3D Generationcited by 36
  4. 2024Score Distillation Sampling with Learned Manifold Correctivecited by 27
  5. 2023SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identitycited by 23
  6. 2025Consistent Flow Distillation for Text-to-3D Generationcited by 17
  7. 2024Robust Symmetry Detection via Riemannian Langevin Dynamicscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Variance Reduction for Expectations with Diffusion Teacherscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Scale-wise Distillation of Diffusion Models
  2. 20261.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillation
  3. 2025Few-Step Diffusion via Score identity Distillation
  4. 2024Multistep Distillation of Diffusion Models via Moment Matching
  5. 2026CrossDistill: Balancing Quality and Diversity via Trajectory-Level Hybrid Few-Step Distillation
  6. 2026Analyzing and Improving Fast Sampling of Text-to-Image Diffusion Models
  7. 2025Learning Few-Step Diffusion Models by Trajectory Distribution Matching
  8. 2024Plug-and-Play Diffusion Distillation