# Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence

arXiv: https://arxiv.org/abs/2506.04700 · HTML: https://mlsanity.com/paper/2506.04700/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [Generalized Sliced Wasserstein Distances](https://mlsanity.com/paper/1902.00434/) (2019)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [Generative Multi-Adversarial Networks](https://mlsanity.com/paper/1611.01673/) (2016)
- [normflows: A PyTorch Package for Normalizing Flows](https://mlsanity.com/paper/2302.12014/) (2023)
- [Pareto GAN: Extending the Representational Power of GANs to Heavy-Tailed Distributions](https://mlsanity.com/paper/2101.09113/) (2021)
- [MCL-GAN: Generative Adversarial Networks with Multiple Specialized Discriminators](https://mlsanity.com/paper/2107.07260/) (2021)
- [Training Implicit Generative Models via an Invariant Statistical Loss](https://mlsanity.com/paper/2402.16435/) (2024)
- [Robust training of implicit generative models for multivariate and heavy-tailed distributions with an invariant statistical loss](https://mlsanity.com/paper/2410.22381/) (2024)

## Built on by (most cited first, 1 of 1)

- [Approximating f-Divergences with Rank Statistics](https://mlsanity.com/paper/2601.22784/) (2026)

## Abstract (arXiv)

Rank-based statistical metrics, such as the invariant statistical loss (ISL), have recently emerged as robust and practically effective tools for training implicit generative models. In this work, we introduce dual-ISL, a novel likelihood-free objective for training implicit generative models that interchanges the roles of the target and model distributions in the ISL framework, yielding a convex optimization problem in the space of model densities. We prove that the resulting rank-based discrepancy dK is i) continuous under weak convergence and with respect to the L1 norm, and ii) convex in its first argument-properties not shared by classical divergences such as KL or Wasserstein distances. Building on this, we develop a theoretical framework that interprets dK as an L2-projection of the density ratio q = p/tilde p onto a Bernstein polynomial basis, from which we derive exact bounds on the truncation error, precise convergence rates, and a closed-form expression for the truncated density approximation. We further extend our analysis to the multivariate setting via random one-dimensional projections, defining a sliced dual-ISL divergence that retains both convexity and continuity. We empirically show that these theoretical advantages translate into practical ones. Specifically, across several benchmarks dual-ISL converges more rapidly, delivers markedly smoother and more stable training, and more effectively prevents mode collapse than classical ISL and other leading implicit generative methods-while also providing an explicit density approximation.