# Pareto GAN: Extending the Representational Power of GANs to Heavy-Tailed Distributions

arXiv: https://arxiv.org/abs/2101.09113 · HTML: https://mlsanity.com/paper/2101.09113/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 22 Jan 2021.

## Builds on (most cited first, 11 of 11)

- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Understanding Deep Neural Networks with Rectified Linear Units](https://mlsanity.com/paper/1611.01491/) (2016)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [Quant GANs: Deep Generation of Financial Time Series](https://mlsanity.com/paper/1907.06673/) (2019)
- [Copula & Marginal Flows: Disentangling the Marginal from its Joint](https://mlsanity.com/paper/1907.03361/) (2019)
- [Robust Super-Resolution GAN, with Manifold-based and Perception Loss](https://mlsanity.com/paper/1903.06920/) (2019)
- [Using GANs for Sharing Networked Time Series Data: Challenges, Initial Promise, and Open Questions](https://mlsanity.com/paper/1909.13403/) (2019)
- [Student's t-Generative Adversarial Networks](https://mlsanity.com/paper/1811.02132/) (2018)

## Built on by (most cited first, 23 of 23)

- [A VAE Approach to Sample Multivariate Extremes](https://mlsanity.com/paper/2306.10987/) (2023)
- [ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models](https://mlsanity.com/paper/2307.00398/) (2023)
- [Beyond the Norm: A Survey of Synthetic Data Generation for Rare Events](https://mlsanity.com/paper/2506.06380/) (2025)
- [Robust Generative Learning with Lipschitz-Regularized α-Divergences Allows Minimal Assumptions on Target Distributions](https://mlsanity.com/paper/2405.13962/) (2024)
- [Generative Machine Learning for Multivariate Angular Simulation](https://mlsanity.com/paper/2504.21505/) (2025)
- [OPT-GAN: A Broad-Spectrum Global Optimizer for Black-box Problems by Learning Distribution](https://mlsanity.com/paper/2102.03888/) (2021)
- [On some theoretical limitations of Generative Adversarial Networks](https://mlsanity.com/paper/2110.10915/) (2021)
- [Combining deep generative models with extreme value theory for synthetic hazard simulation: a multivariate and spatially coherent approach](https://mlsanity.com/paper/2311.18521/) (2023)
- [Robust training of implicit generative models for multivariate and heavy-tailed distributions with an invariant statistical loss](https://mlsanity.com/paper/2410.22381/) (2024)
- [Data-Driven Modeling of Noise Time Series with Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/2207.01110/) (2022)
- [Simulation of Multivariate Extremes: a Wasserstein-Aitchison GAN approach](https://mlsanity.com/paper/2504.21438/) (2025)
- [Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence](https://mlsanity.com/paper/2506.04700/) (2025)
- [Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows](https://mlsanity.com/paper/2608.11544/) (2026)
- [Approximating Probability Distributions by using Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/2103.10060/) (2021)
- [Modelling and simulating spatial extremes by combining extreme value theory with generative adversarial networks](https://mlsanity.com/paper/2111.00267/) (2021)
- [Generalized Regression with Conditional GANs](https://mlsanity.com/paper/2404.13500/) (2024)
- [Capturing Unseen Spatial Heat Extremes Through Dependence-Aware Generative Modeling](https://mlsanity.com/paper/2507.09211/) (2025)
- [Generative Adversarial Regression (GAR): Learning Conditional Risk Scenarios](https://mlsanity.com/paper/2603.08553/) (2026)
- [An Extreme Value Perspective on Learning Stress Laws](https://mlsanity.com/paper/2607.10700/) (2026)
- [Physics-Aware Conditional SetGAN for Spatially Consistent Multi-User TR 38.901 Channel Generation](https://mlsanity.com/paper/2607.11429/) (2026)
- [TailBooster: A Dual-Layer Generative Framework for Extreme Value Augmentation with Operational Validity Enforcement](https://mlsanity.com/paper/2608.11951/) (2026)
- [Financial Tail Risk Beyond Lipschitz Continuity via Semi-Discrete Optimal Transport](https://mlsanity.com/paper/2609.27785/) (2026)
- [Radial Tail Repair: Post-Training Correction for Diffusion and Flow Models](https://mlsanity.com/paper/2610.16851/) (2026)

## Abstract (arXiv)

Generative adversarial networks (GANs) are often billed as "universal distribution learners", but precisely what distributions they can represent and learn is still an open question. Heavy-tailed distributions are prevalent in many different domains such as financial risk-assessment, physics, and epidemiology. We observe that existing GAN architectures do a poor job of matching the asymptotic behavior of heavy-tailed distributions, a problem that we show stems from their construction. Additionally, when faced with the infinite moments and large distances between outlier points that are characteristic of heavy-tailed distributions, common loss functions produce unstable or near-zero gradients. We address these problems with the Pareto GAN. A Pareto GAN leverages extreme value theory and the functional properties of neural networks to learn a distribution that matches the asymptotic behavior of the marginal distributions of the features. We identify issues with standard loss functions and propose the use of alternative metric spaces that enable stable and efficient learning. Finally, we evaluate our proposed approach on a variety of heavy-tailed datasets.