# Weighted quantization using MMD: From mean field to mean shift via gradient flows

arXiv: https://arxiv.org/abs/2502.10600 · HTML: https://mlsanity.com/paper/2502.10600/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 96 references that are in the map.
- First posted on arXiv 14 Feb 2025.

## Builds on (most cited first, 30 of 45)

- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measures](https://mlsanity.com/paper/1508.07941/) (2015)
- [On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions](https://mlsanity.com/paper/1502.06800/) (2015)
- [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020)
- [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021)
- [Sparse Optimization on Measures with Over-parameterized Gradient Descent](https://mlsanity.com/paper/1907.10300/) (2019)
- [Kernel Thinning](https://mlsanity.com/paper/2105.05842/) (2021)
- [Statistical optimal transport](https://mlsanity.com/paper/2407.18163/) (2024)
- [Accelerating Langevin Sampling with Birth-death](https://mlsanity.com/paper/1905.09863/) (2019)
- [Stein Points](https://mlsanity.com/paper/1803.10161/) (2018)
- [KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support](https://mlsanity.com/paper/2106.08929/) (2021)
- [Generative Sliced MMD Flows with Riesz Kernels](https://mlsanity.com/paper/2305.11463/) (2023)
- [Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics](https://mlsanity.com/paper/2211.00450/) (2022)
- [Generalized Kernel Thinning](https://mlsanity.com/paper/2110.01593/) (2021)
- [Kernel quadrature with DPPs](https://mlsanity.com/paper/1906.07832/) (2019)
- [Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2301.01766/) (2023)
- [Sampling in Unit Time with Kernel Fisher-Rao Flow](https://mlsanity.com/paper/2401.03892/) (2024)
- [Optimal Thinning of MCMC Output](https://mlsanity.com/paper/2005.03952/) (2020)
- [Positively Weighted Kernel Quadrature via Subsampling](https://mlsanity.com/paper/2107.09597/) (2021)
- [Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering](https://mlsanity.com/paper/1501.02056/) (2015)
- [Global convergence of neuron birth-death dynamics](https://mlsanity.com/paper/1902.01843/) (2019)
- [Kernel interpolation with continuous volume sampling](https://mlsanity.com/paper/2002.09677/) (2020)
- [Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees](https://mlsanity.com/paper/1506.02681/) (2015)
- [A review of mean-shift algorithms for clustering](https://mlsanity.com/paper/1503.00687/) (2015)

## Built on by (most cited first, 12 of 12)

- [Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences](https://mlsanity.com/paper/2603.10592/) (2026)
- [A Dictionary of Closed-Form Kernel Mean Embeddings](https://mlsanity.com/paper/2504.18830/) (2025)
- [A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows](https://mlsanity.com/paper/2507.04330/) (2025)
- [Stationary MMD Points](https://mlsanity.com/paper/2505.20754/) (2025)
- [To discretize continually: Mean shift interacting particle systems for Bayesian inference](https://mlsanity.com/paper/2605.14142/) (2026)
- [Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD](https://mlsanity.com/paper/2607.24235/) (2026)
- [A Gradient Flow Perspective on Minimum MMD Estimation](https://mlsanity.com/paper/2607.03871/) (2026)
- [Amortized quadrature for posterior expectations in inverse problems](https://mlsanity.com/paper/2606.15871/) (2026)
- [A nonparametric two-sample test using a parametric integral probability metric](https://mlsanity.com/paper/2606.16941/) (2026)
- [Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization](https://mlsanity.com/paper/2606.27767/) (2026)
- [Kernel-based External Validity of Experimental Studies](https://mlsanity.com/paper/2610.13625/) (2026)
- [On the Optimisation Landscape of Overparameterised Weighted MMD](https://mlsanity.com/paper/2610.19861/) (2026)

## Abstract (arXiv)

Approximating a probability distribution using a set of particles is a fundamental problem in machine learning and statistics, with applications including clustering and quantization. Formally, we seek a weighted mixture of Dirac measures that best approximates the target distribution. While much existing work relies on the Wasserstein distance to quantify approximation errors, maximum mean discrepancy (MMD) has received comparatively less attention, especially when allowing for variable particle weights. We argue that a Wasserstein-Fisher-Rao gradient flow is well-suited for designing quantizations optimal under MMD. We show that a system of interacting particles satisfying a set of ODEs discretizes this flow. We further derive a new fixed-point algorithm called mean shift interacting particles (MSIP). We show that MSIP extends the classical mean shift algorithm, widely used for identifying modes in kernel density estimators. Moreover, we show that MSIP can be interpreted as preconditioned gradient descent and that it acts as a relaxation of Lloyd's algorithm for clustering. Our unification of gradient flows, mean shift, and MMD-optimal quantization yields algorithms that are more robust than state-of-the-art methods, as demonstrated via high-dimensional and multi-modal numerical experiments.