# How does Lipschitz Regularization Influence GAN Training?

arXiv: https://arxiv.org/abs/1811.09567 · HTML: https://mlsanity.com/paper/1811.09567/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2018.

## Built on by (most cited first, 6 of 6)

- [E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion](https://mlsanity.com/paper/2104.07661/) (2021)
- [Some Fundamental Aspects about Lipschitz Continuity of Neural Networks](https://mlsanity.com/paper/2302.10886/) (2023)
- [Detection of AI Deepfake and Fraud in Online Payments Using GAN-Based Models](https://mlsanity.com/paper/2501.07033/) (2025)
- [MiDAS: Multi-integrated Domain Adaptive Supervision for Fake News Detection](https://mlsanity.com/paper/2205.09817/) (2022)
- [Exploring and Exploiting Hubness Priors for High-Quality GAN Latent Sampling](https://mlsanity.com/paper/2206.06014/) (2022)
- [Feature Proliferation -- the "Cancer" in StyleGAN and its Treatments](https://mlsanity.com/paper/2310.08921/) (2023)

## Abstract (arXiv)

Despite the success of Lipschitz regularization in stabilizing GAN training, the exact reason of its effectiveness remains poorly understood. The direct effect of K-Lipschitz regularization is to restrict the L2-norm of the neural network gradient to be smaller than a threshold K (e.g., K=1) such that |nabla f| ≤ K. In this work, we uncover an even more important effect of Lipschitz regularization by examining its impact on the loss function: It degenerates GAN loss functions to almost linear ones by restricting their domain and interval of attainable gradient values. Our analysis shows that loss functions are only successful if they are degenerated to almost linear ones. We also show that loss functions perform poorly if they are not degenerated and that a wide range of functions can be used as loss function as long as they are sufficiently degenerated by regularization. Basically, Lipschitz regularization ensures that all loss functions effectively work in the same way. Empirically, we verify our proposition on the MNIST, CIFAR10 and CelebA datasets.