# Training generative neural networks via Maximum Mean Discrepancy optimization

arXiv: https://arxiv.org/abs/1505.03906 · HTML: https://mlsanity.com/paper/1505.03906/

## Key facts

- Cited by 151 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2015.

## Built on by (most cited first, 30 of 151)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [Generative Modeling via Drifting](https://mlsanity.com/paper/2602.04770/) (2026)
- [Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation](https://mlsanity.com/paper/2406.02347/) (2024)
- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Robust Bayesian Inference for Simulator-based Models via the MMD Posterior Bootstrap](https://mlsanity.com/paper/2202.04744/) (2022)
- [KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support](https://mlsanity.com/paper/2106.08929/) (2021)
- [Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization](https://mlsanity.com/paper/2112.08217/) (2021)
- [SizeShiftReg: a Regularization Method for Improving Size-Generalization in Graph Neural Networks](https://mlsanity.com/paper/2207.07888/) (2022)
- [Evading DeepFake Detectors via Adversarial Statistical Consistency](https://mlsanity.com/paper/2304.11670/) (2023)
- [Generative Sliced MMD Flows with Riesz Kernels](https://mlsanity.com/paper/2305.11463/) (2023)
- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM](https://mlsanity.com/paper/2206.08873/) (2022)
- [On Error Propagation of Diffusion Models](https://mlsanity.com/paper/2308.05021/) (2023)
- [DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems](https://mlsanity.com/paper/2402.04467/) (2024)
- [A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful](https://mlsanity.com/paper/2110.06581/) (2021)
- [Pre-trained Perceptual Features Improve Differentially Private Image Generation](https://mlsanity.com/paper/2205.12900/) (2022)
- [Generative machine learning methods for multivariate ensemble post-processing](https://mlsanity.com/paper/2211.01345/) (2022)
- [Distributional Diffusion Models with Scoring Rules](https://mlsanity.com/paper/2502.02483/) (2025)
- [Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks](https://mlsanity.com/paper/2306.13103/) (2023)
- [MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining](https://mlsanity.com/paper/2110.08009/) (2021)
- [Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance](https://mlsanity.com/paper/2509.02055/) (2025)
- [On the capacity of deep generative networks for approximating distributions](https://mlsanity.com/paper/2101.12353/) (2021)
- [Domain-Adversarial Training of Self-Attention Based Networks for Land Cover Classification using Multi-temporal Sentinel-2 Satellite Imagery](https://mlsanity.com/paper/2104.00564/) (2021)
- [Towards Robust Graph Incremental Learning on Evolving Graphs](https://mlsanity.com/paper/2402.12987/) (2024)
- [Proper scoring rules for estimation and forecast evaluation](https://mlsanity.com/paper/2504.01781/) (2025)
- [Implicit Generative Copulas](https://mlsanity.com/paper/2109.14567/) (2021)
- [Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences](https://mlsanity.com/paper/2603.10592/) (2026)
- [Composite Goodness-of-fit Tests with Kernels](https://mlsanity.com/paper/2111.10275/) (2021)

## Abstract (arXiv)

We consider training a deep neural network to generate samples from an unknown distribution given i.i.d. data. We frame learning as an optimization minimizing a two-sample test statistic---informally speaking, a good generator network produces samples that cause a two-sample test to fail to reject the null hypothesis. As our two-sample test statistic, we use an unbiased estimate of the maximum mean discrepancy, which is the centerpiece of the nonparametric kernel two-sample test proposed by Gretton et al. (2012). We compare to the adversarial nets framework introduced by Goodfellow et al. (2014), in which learning is a two-player game between a generator network and an adversarial discriminator network, both trained to outwit the other. From this perspective, the MMD statistic plays the role of the discriminator. In addition to empirical comparisons, we prove bounds on the generalization error incurred by optimizing the empirical MMD.