# On the regularization of Wasserstein GANs

arXiv: https://arxiv.org/abs/1709.08894 · HTML: https://mlsanity.com/paper/1709.08894/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2017.

## Built on by (most cited first, 30 of 65)

- [Neural Optimal Transport](https://mlsanity.com/paper/2201.12220/) (2022)
- [SPECTRE: Spectral Conditioning Helps to Overcome the Expressivity Limits of One-shot Graph Generators](https://mlsanity.com/paper/2204.01613/) (2022)
- [Generative Modeling with Optimal Transport Maps](https://mlsanity.com/paper/2110.02999/) (2021)
- [Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport](https://mlsanity.com/paper/2305.14777/) (2023)
- [Sinkhorn Distributionally Robust Optimization](https://mlsanity.com/paper/2109.11926/) (2021)
- [StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis](https://mlsanity.com/paper/2206.09479/) (2022)
- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [Neural Optimal Transport with General Cost Functionals](https://mlsanity.com/paper/2205.15403/) (2022)
- [Kernel Neural Optimal Transport](https://mlsanity.com/paper/2205.15269/) (2022)
- [On amortizing convex conjugates for optimal transport](https://mlsanity.com/paper/2210.12153/) (2022)
- [Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey](https://mlsanity.com/paper/2305.05080/) (2023)
- [Some Fundamental Aspects about Lipschitz Continuity of Neural Networks](https://mlsanity.com/paper/2302.10886/) (2023)
- [Disentangled Recurrent Wasserstein Autoencoder](https://mlsanity.com/paper/2101.07496/) (2021)
- [Approximation bounds for norm constrained neural networks with applications to regression and GANs](https://mlsanity.com/paper/2201.09418/) (2022)
- [A Representative Study on Human Detection of Artificially Generated Media Across Countries](https://mlsanity.com/paper/2312.05976/) (2023)
- [Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?](https://mlsanity.com/paper/2206.07767/) (2022)
- [Wasserstein Proximal of GANs](https://mlsanity.com/paper/2102.06862/) (2021)
- [Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training](https://mlsanity.com/paper/2103.00673/) (2021)
- [Learning quantum data with the quantum Earth Mover's distance](https://mlsanity.com/paper/2101.03037/) (2021)
- [Understanding Diffusion Models by Feynman's Path Integral](https://mlsanity.com/paper/2403.11262/) (2024)
- [Self-Supervised GANs with Label Augmentation](https://mlsanity.com/paper/2106.08601/) (2021)
- [Generalization Error of GAN from the Discriminator's Perspective](https://mlsanity.com/paper/2107.03633/) (2021)
- [Analyzing and Improving Optimal-Transport-based Adversarial Networks](https://mlsanity.com/paper/2310.02611/) (2023)
- [Wasserstein GANs with Gradient Penalty Compute Congested Transport](https://mlsanity.com/paper/2109.00528/) (2021)
- [Provably Convergent Policy Optimization via Metric-aware Trust Region Methods](https://mlsanity.com/paper/2306.14133/) (2023)
- [Moreau-Yosida f-divergences](https://mlsanity.com/paper/2102.13416/) (2021)
- [On the Existence of Optimal Transport Gradient for Learning Generative Models](https://mlsanity.com/paper/2102.05542/) (2021)
- [PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier](https://mlsanity.com/paper/2103.15469/) (2021)
- [Input Invex Neural Network](https://mlsanity.com/paper/2106.08748/) (2021)
- [Detecting data-driven robust statistical arbitrage strategies with deep neural networks](https://mlsanity.com/paper/2203.03179/) (2022)

## Abstract (arXiv)

Since their invention, generative adversarial networks (GANs) have become a popular approach for learning to model a distribution of real (unlabeled) data. Convergence problems during training are overcome by Wasserstein GANs which minimize the distance between the model and the empirical distribution in terms of a different metric, but thereby introduce a Lipschitz constraint into the optimization problem. A simple way to enforce the Lipschitz constraint on the class of functions, which can be modeled by the neural network, is weight clipping. It was proposed that training can be improved by instead augmenting the loss by a regularization term that penalizes the deviation of the gradient of the critic (as a function of the network's input) from one. We present theoretical arguments why using a weaker regularization term enforcing the Lipschitz constraint is preferable. These arguments are supported by experimental results on toy data sets.