# Robust Generative Learning with Lipschitz-Regularized α-Divergences Allows Minimal Assumptions on Target Distributions

arXiv: https://arxiv.org/abs/2405.13962 · HTML: https://mlsanity.com/paper/2405.13962/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 56 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 26 of 26)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport](https://mlsanity.com/paper/2006.00104/) (2020)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [Quant GANs: Deep Generation of Financial Time Series](https://mlsanity.com/paper/1907.06673/) (2019)
- [Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem](https://mlsanity.com/paper/1905.11882/) (2019)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [A mean-field games laboratory for generative modeling](https://mlsanity.com/paper/2304.13534/) (2023)
- [Practical and Consistent Estimation of f-Divergences](https://mlsanity.com/paper/1905.11112/) (2019)
- [Pareto GAN: Extending the Representational Power of GANs to Heavy-Tailed Distributions](https://mlsanity.com/paper/2101.09113/) (2021)
- [Distribution learning via neural differential equations: a nonparametric statistical perspective](https://mlsanity.com/paper/2309.01043/) (2023)
- [Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference](https://mlsanity.com/paper/2310.16975/) (2023)
- [Structure-preserving GANs](https://mlsanity.com/paper/2202.01129/) (2022)
- [Non-Asymptotic Performance Guarantees for Neural Estimation of mathsff-Divergences](https://mlsanity.com/paper/2103.06923/) (2021)
- [Sample Complexity of Probability Divergences under Group Symmetry](https://mlsanity.com/paper/2302.01915/) (2023)
- [Sample Complexity Bounds for Estimating Probability Divergences under Invariances](https://mlsanity.com/paper/2311.02868/) (2023)
- [Formulation and properties of a divergence used to compare probability measures without absolute continuity](https://mlsanity.com/paper/1911.07422/) (2019)
- [Lipschitz-regularized gradient flows and generative particle algorithms for high-dimensional scarce data](https://mlsanity.com/paper/2210.17230/) (2022)
- [Multivariate f-Divergence Estimation With Confidence](https://mlsanity.com/paper/1411.2045/) (2014)
- [Closed-loop adaptive control of extreme events in a turbulent flow](https://mlsanity.com/paper/1906.03042/) (2019)

## Built on by (most cited first, 4 of 4)

- [Robustness and Structure Preservation in Flow-Based Generative Models via Wasserstein Path-Space Divergences](https://mlsanity.com/paper/2410.01244/) (2024)
- [Proximal optimal transport divergences](https://mlsanity.com/paper/2505.12097/) (2025)
- [Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows](https://mlsanity.com/paper/2608.11544/) (2026)
- [Graph-Informed Adversarial Modeling: Infimal Subadditivity of Interpolative Divergences](https://mlsanity.com/paper/2603.20025/) (2026)

## Abstract (arXiv)

This paper demonstrates the robustness of Lipschitz-regularized α-divergences as objective functionals in generative modeling, showing they enable stable learning across a wide range of target distributions with minimal assumptions. We establish that these divergences remain finite under a mild condition-that the source distribution has a finite first moment-regardless of the properties of the target distribution, making them adaptable to the structure of target distributions. Furthermore, we prove the existence and finiteness of their variational derivatives, which are essential for stable training of generative models such as GANs and gradient flows. For heavy-tailed targets, we derive necessary and sufficient conditions that connect data dimension, α, and tail behavior to divergence finiteness, that also provide insights into the selection of suitable α's. We also provide the first sample complexity bounds for empirical estimations of these divergences on unbounded domains. As a byproduct, we obtain the first sample complexity bounds for empirical estimations of these divergences and the Wasserstein-1 metric with group symmetry on unbounded domains. Numerical experiments confirm that generative models leveraging Lipschitz-regularized α-divergences can stably learn distributions in various challenging scenarios, including those with heavy tails or complex, low-dimensional, or fractal support, all without any prior knowledge of the structure of target distributions.