# Understanding Entropic Regularization in GANs

arXiv: https://arxiv.org/abs/2111.01387 · HTML: https://mlsanity.com/paper/2111.01387/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Learning Generative Models with Sinkhorn Divergences](https://mlsanity.com/paper/1706.00292/) (2017)
- [Wasserstein-2 Generative Networks](https://mlsanity.com/paper/1909.13082/) (2019)
- [Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem](https://mlsanity.com/paper/1905.11882/) (2019)
- [Smooth and Sparse Optimal Transport](https://mlsanity.com/paper/1710.06276/) (2017)
- [On the Convergence and Robustness of Training GANs with Regularized Optimal Transport](https://mlsanity.com/paper/1802.08249/) (2018)
- [Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect](https://mlsanity.com/paper/1803.01541/) (2018)
- [Learning and Forecasting Opinion Dynamics in Social Networks](https://mlsanity.com/paper/1506.05474/) (2015)
- [Generalization Properties of Optimal Transport GANs with Latent Distribution Learning](https://mlsanity.com/paper/2007.14641/) (2020)
- [Understanding GANs: the LQG Setting](https://mlsanity.com/paper/1710.10793/) (2017)
- [Entropic GANs meet VAEs: A Statistical Approach to Compute Sample Likelihoods in GANs](https://mlsanity.com/paper/1810.04147/) (2018)

## Built on by (most cited first, 6 of 6)

- [Addressing GAN Training Instabilities via Tunable Classification Losses](https://mlsanity.com/paper/2310.18291/) (2023)
- [One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Models](https://mlsanity.com/paper/2305.18900/) (2023)
- [Training generative models from privatized data](https://mlsanity.com/paper/2306.09547/) (2023)
- [Generalizing Machine Learning Evaluation through the Integration of Shannon Entropy and Rough Set Theory](https://mlsanity.com/paper/2404.12511/) (2024)
- [Ranked Set Sampling-Based Multilayer Perceptron: Improving Generalization via Variance-Based Bounds](https://mlsanity.com/paper/2507.08465/) (2025)
- [On optimal solutions of classical and sliced Wasserstein GANs with non-Gaussian data](https://mlsanity.com/paper/2509.06505/) (2025)

## Abstract (arXiv)

Generative Adversarial Networks are a popular method for learning distributions from data by modeling the target distribution as a function of a known distribution. The function, often referred to as the generator, is optimized to minimize a chosen distance measure between the generated and target distributions. One commonly used measure for this purpose is the Wasserstein distance. However, Wasserstein distance is hard to compute and optimize, and in practice entropic regularization techniques are used to improve numerical convergence. The influence of regularization on the learned solution, however, remains not well-understood. In this paper, we study how several popular entropic regularizations of Wasserstein distance impact the solution in a simple benchmark setting where the generator is linear and the target distribution is high-dimensional Gaussian. We show that entropy regularization promotes the solution sparsification, while replacing the Wasserstein distance with the Sinkhorn divergence recovers the unregularized solution. Both regularization techniques remove the curse of dimensionality suffered by Wasserstein distance. We show that the optimal generator can be learned to accuracy ε with O(1/ε2) samples from the target distribution. We thus conclude that these regularization techniques can improve the quality of the generator learned from empirical data for a large class of distributions.