# Non-parametric estimation of Jensen-Shannon Divergence in Generative Adversarial Network training

arXiv: https://arxiv.org/abs/1705.09199 · HTML: https://mlsanity.com/paper/1705.09199/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2017.

## Built on by (most cited first, 6 of 6)

- [Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach](https://mlsanity.com/paper/2502.00577/) (2025)
- [Matching aggregate posteriors in the variational autoencoder](https://mlsanity.com/paper/2311.07693/) (2023)
- [Bt-GAN: Generating Fair Synthetic Healthdata via Bias-transforming Generative Adversarial Networks](https://mlsanity.com/paper/2404.13634/) (2024)
- [Towards A Multi-agent System for Online Hate Speech Detection](https://mlsanity.com/paper/2105.01129/) (2021)
- [KLIEP-based Density Ratio Estimation for Semantically Consistent Synthetic to Real Images Adaptation in Urban Traffic Scenes](https://mlsanity.com/paper/2105.12549/) (2021)
- [EVHA: Explainable Vision System for Hardware Testing and Assurance -- An Overview](https://mlsanity.com/paper/2207.09627/) (2022)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have become a widely popular framework for generative modelling of high-dimensional datasets. However their training is well-known to be difficult. This work presents a rigorous statistical analysis of GANs providing straight-forward explanations for common training pathologies such as vanishing gradients. Furthermore, it proposes a new training objective, Kernel GANs, and demonstrates its practical effectiveness on large-scale real-world data sets. A key element in the analysis is the distinction between training with respect to the (unknown) data distribution, and its empirical counterpart. To overcome issues in GAN training, we pursue the idea of smoothing the Jensen-Shannon Divergence (JSD) by incorporating noise in the input distributions of the discriminator. As we show, this effectively leads to an empirical version of the JSD in which the true and the generator densities are replaced by kernel density estimates, which leads to Kernel GANs.