# Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)

arXiv: https://arxiv.org/abs/2103.01678 · HTML: https://mlsanity.com/paper/2103.01678/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Learning Generative Models with Sinkhorn Divergences](https://mlsanity.com/paper/1706.00292/) (2017)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [BEGAN: Boundary Equilibrium Generative Adversarial Networks](https://mlsanity.com/paper/1703.10717/) (2017)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [Generative Modeling using the Sliced Wasserstein Distance](https://mlsanity.com/paper/1803.11188/) (2018)
- [Adversarial Regularizers in Inverse Problems](https://mlsanity.com/paper/1805.11572/) (2018)
- [Subspace Robust Wasserstein Distances](https://mlsanity.com/paper/1901.08949/) (2019)
- [Learning with minibatch Wasserstein : asymptotic and gradient properties](https://mlsanity.com/paper/1910.04091/) (2019)
- [Inductive Bias of Deep Convolutional Networks through Pooling Geometry](https://mlsanity.com/paper/1605.06743/) (2016)
- [Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step](https://mlsanity.com/paper/1710.08446/) (2017)
- [A Geometric View of Optimal Transportation and Generative Model](https://mlsanity.com/paper/1710.05488/) (2017)
- [How Well Do WGANs Estimate the Wasserstein Metric?](https://mlsanity.com/paper/1910.03875/) (2019)
- [Implicit competitive regularization in GANs](https://mlsanity.com/paper/1910.05852/) (2019)
- [A gradual, semi-discrete approach to generative network training via explicit Wasserstein minimization](https://mlsanity.com/paper/1906.03471/) (2019)
- [(q,p)-Wasserstein GANs: Comparing Ground Metrics for Wasserstein GANs](https://mlsanity.com/paper/1902.03642/) (2019)
- [On the estimation of the Wasserstein distance in generative models](https://mlsanity.com/paper/1910.00888/) (2019)
- [Wasserstein GAN Can Perform PCA](https://mlsanity.com/paper/1902.09073/) (2019)

## Built on by (most cited first, 30 of 35)

- [An Introduction to Deep Generative Modeling](https://mlsanity.com/paper/2103.05180/) (2021)
- [Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark](https://mlsanity.com/paper/2106.01954/) (2021)
- [Optimal Flow Matching: Learning Straight Trajectories in Just One Step](https://mlsanity.com/paper/2403.13117/) (2024)
- [Recent Advances in Optimal Transport for Machine Learning](https://mlsanity.com/paper/2306.16156/) (2023)
- [UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation](https://mlsanity.com/paper/2203.02557/) (2022)
- [Improving Mini-batch Optimal Transport via Partial Transportation](https://mlsanity.com/paper/2108.09645/) (2021)
- [Adversarial Intrinsic Motivation for Reinforcement Learning](https://mlsanity.com/paper/2105.13345/) (2021)
- [Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey](https://mlsanity.com/paper/2305.05080/) (2023)
- [Regularising Inverse Problems with Generative Machine Learning Models](https://mlsanity.com/paper/2107.11191/) (2021)
- [Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?](https://mlsanity.com/paper/2206.07767/) (2022)
- [Amortized Projection Optimization for Sliced Wasserstein Generative Models](https://mlsanity.com/paper/2203.13417/) (2022)
- [A survey and taxonomy of loss functions in machine learning](https://mlsanity.com/paper/2301.05579/) (2023)
- [Adversarial Score Distillation: When score distillation meets GAN](https://mlsanity.com/paper/2312.00739/) (2023)
- [Wasserstein Patch Prior for Image Superresolution](https://mlsanity.com/paper/2109.12880/) (2021)
- [Time-Causal VAE: Robust Financial Time Series Generator](https://mlsanity.com/paper/2411.02947/) (2024)
- [Imitation Learning from Observation through Optimal Transport](https://mlsanity.com/paper/2310.01632/) (2023)
- [Wasserstein GANs with Gradient Penalty Compute Congested Transport](https://mlsanity.com/paper/2109.00528/) (2021)
- [Offline Imitation from Observation via Primal Wasserstein State Occupancy Matching](https://mlsanity.com/paper/2311.01331/) (2023)
- [Slicing Wasserstein Over Wasserstein Via Functional Optimal Transport](https://mlsanity.com/paper/2509.22138/) (2025)
- [Learning Polynomial Transformations](https://mlsanity.com/paper/2204.04209/) (2022)
- [Approximation Theory, Computing, and Deep Learning on the Wasserstein Space](https://mlsanity.com/paper/2310.19548/) (2023)
- [Manifold-preserved GANs](https://mlsanity.com/paper/2109.08955/) (2021)
- [Variational Diffusion Auto-encoder: Latent Space Extraction from Pre-trained Diffusion Models](https://mlsanity.com/paper/2304.12141/) (2023)
- [Deep Generative Quantile Bayes](https://mlsanity.com/paper/2410.08378/) (2024)
- [Rethinking Multidimensional Discriminator Output for Generative Adversarial Networks](https://mlsanity.com/paper/2109.03378/) (2021)
- [Adversarial Reweighting Guided by Wasserstein Distance for Bias Mitigation](https://mlsanity.com/paper/2311.12684/) (2023)
- [Global Optimisation of Black-Box Functions with Generative Models in the Wasserstein Space](https://mlsanity.com/paper/2407.11917/) (2024)
- [Variational Neural Stochastic Differential Equations with Change Points](https://mlsanity.com/paper/2411.00635/) (2024)
- [Learning to Emulate Chaos: Adversarial Optimal Transport Regularization](https://mlsanity.com/paper/2604.21097/) (2026)
- [Insights into Closed-form IPM-GAN Discriminator Guidance for Diffusion Modeling](https://mlsanity.com/paper/2306.01654/) (2023)

## Abstract (arXiv)

Wasserstein GANs are based on the idea of minimising the Wasserstein distance between a real and a generated distribution. We provide an in-depth mathematical analysis of differences between the theoretical setup and the reality of training Wasserstein GANs. In this work, we gather both theoretical and empirical evidence that the WGAN loss is not a meaningful approximation of the Wasserstein distance. Moreover, we argue that the Wasserstein distance is not even a desirable loss function for deep generative models, and conclude that the success of Wasserstein GANs can in truth be attributed to a failure to approximate the Wasserstein distance.