# On the Optimization Landscape of Maximum Mean Discrepancy

arXiv: https://arxiv.org/abs/2110.13452 · HTML: https://mlsanity.com/paper/2110.13452/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 22 of 22)

- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [How to Escape Saddle Points Efficiently](https://mlsanity.com/paper/1703.00887/) (2017)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019)
- [The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization](https://mlsanity.com/paper/1807.03907/) (2018)
- [Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy](https://mlsanity.com/paper/1611.04488/) (2016)
- [The Landscape of Empirical Risk for Non-convex Losses](https://mlsanity.com/paper/1607.06534/) (2016)
- [Beyond Convexity: Stochastic Quasi-Convex Optimization](https://mlsanity.com/paper/1507.02030/) (2015)
- [Ten Steps of EM Suffice for Mixtures of Two Gaussians](https://mlsanity.com/paper/1609.00368/) (2016)
- [Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences](https://mlsanity.com/paper/1609.00978/) (2016)
- [When Are Nonconvex Problems Not Scary?](https://mlsanity.com/paper/1510.06096/) (2015)
- [Improving MMD-GAN Training with Repulsive Loss Function](https://mlsanity.com/paper/1812.09916/) (2018)
- [On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow](https://mlsanity.com/paper/2011.02402/) (2020)
- [SGD Learns One-Layer Networks in WGANs](https://mlsanity.com/paper/1910.07030/) (2019)
- [Understanding GANs: the LQG Setting](https://mlsanity.com/paper/1710.10793/) (2017)
- [Closed-form Expressions for Maximum Mean Discrepancy with Applications to Wasserstein Auto-Encoders](https://mlsanity.com/paper/1901.03227/) (2019)
- [Minimax Estimation of Neural Net Distance](https://mlsanity.com/paper/1811.01054/) (2018)
- [Wasserstein GAN Can Perform PCA](https://mlsanity.com/paper/1902.09073/) (2019)
- [GAT-GMM: Generative Adversarial Training for Gaussian Mixture Models](https://mlsanity.com/paper/2006.10293/) (2020)

## Built on by (most cited first, 1 of 1)

- [On the Optimisation Landscape of Overparameterised Weighted MMD](https://mlsanity.com/paper/2610.19861/) (2026)

## Abstract (arXiv)

Generative models have been successfully used for generating realistic signals. Because the likelihood function is typically intractable in most of these models, the common practice is to use "implicit" models that avoid likelihood calculation. However, it is hard to obtain theoretical guarantees for such models. In particular, it is not understood when they can globally optimize their non-convex objectives. Here we provide such an analysis for the case of Maximum Mean Discrepancy (MMD) learning of generative models. We prove several optimality results, including for a Gaussian distribution with low rank covariance (where likelihood is inapplicable) and a mixture of Gaussians. Our analysis shows that that the MMD optimization landscape is benign in these cases, and therefore gradient based methods will globally minimize the MMD objective.