# Improving MMD-GAN Training with Repulsive Loss Function

arXiv: https://arxiv.org/abs/1812.09916 · HTML: https://mlsanity.com/paper/1812.09916/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2018.

## Built on by (most cited first, 22 of 22)

- [Evading DeepFake Detectors via Adversarial Statistical Consistency](https://mlsanity.com/paper/2304.11670/) (2023)
- [Trainability barriers and opportunities in quantum generative modeling](https://mlsanity.com/paper/2305.02881/) (2023)
- [Styleformer: Transformer based Generative Adversarial Networks with Style Vector](https://mlsanity.com/paper/2106.07023/) (2021)
- [Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusion](https://mlsanity.com/paper/2405.00228/) (2024)
- [Scale-wise Distillation of Diffusion Models](https://mlsanity.com/paper/2503.16397/) (2025)
- [Partition-Guided GANs](https://mlsanity.com/paper/2104.00816/) (2021)
- [Neural Sinkhorn Gradient Flow](https://mlsanity.com/paper/2401.14069/) (2024)
- [Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations](https://mlsanity.com/paper/2203.01933/) (2022)
- [Dynamically Grown Generative Adversarial Networks](https://mlsanity.com/paper/2106.08505/) (2021)
- [EAGAN: Efficient Two-stage Evolutionary Architecture Search for GANs](https://mlsanity.com/paper/2111.15097/) (2021)
- [Can We Find Strong Lottery Tickets in Generative Models?](https://mlsanity.com/paper/2212.08311/) (2022)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Energy-Calibrated VAE with Test Time Free Lunch](https://mlsanity.com/paper/2311.04071/) (2023)
- [Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernels](https://mlsanity.com/paper/2105.04538/) (2021)
- [Data Interpolants -- That's What Discriminators in Higher-order Gradient-regularized GANs Are](https://mlsanity.com/paper/2306.00785/) (2023)
- [On the Optimization Landscape of Maximum Mean Discrepancy](https://mlsanity.com/paper/2110.13452/) (2021)
- [Elastic Interaction Energy-Based Generative Model: Approximation in Feature Space](https://mlsanity.com/paper/2303.10553/) (2023)
- [Cross-Domain Policy Transfer by Representation Alignment via Multi-Domain Behavioral Cloning](https://mlsanity.com/paper/2407.16912/) (2024)
- [CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics](https://mlsanity.com/paper/2504.05945/) (2025)
- [RCC-GAN: Regularized Compound Conditional GAN for Large-Scale Tabular Data Synthesis](https://mlsanity.com/paper/2205.11693/) (2022)
- [Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flow](https://mlsanity.com/paper/2307.01879/) (2023)
- [Arbitrary Distributions Mapping via SyMOT-Flow: A Flow-based Approach Integrating Maximum Mean Discrepancy and Optimal Transport](https://mlsanity.com/paper/2308.13815/) (2023)

## Abstract (arXiv)

Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget. This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions. First, we argue that the existing MMD loss function may discourage the learning of fine details in data as it attempts to contract the discriminator outputs of real data. To address this issue, we propose a repulsive loss function to actively learn the difference among the real data by simply rearranging the terms in MMD. Second, inspired by the hinge loss, we propose a bounded Gaussian kernel to stabilize the training of MMD-GAN with the repulsive loss function. The proposed methods are applied to the unsupervised image generation tasks on CIFAR-10, STL-10, CelebA, and LSUN bedroom datasets. Results show that the repulsive loss function significantly improves over the MMD loss at no additional computational cost and outperforms other representative loss functions. The proposed methods achieve an FID score of 16.21 on the CIFAR-10 dataset using a single DCGAN network and spectral normalization.