# Subsampling Generative Adversarial Networks: Density Ratio Estimation in Feature Space with Softplus Loss

arXiv: https://arxiv.org/abs/1909.10670 · HTML: https://mlsanity.com/paper/1909.10670/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2019.

## Built on by (most cited first, 11 of 11)

- [Generating High Fidelity Data from Low-density Regions using Diffusion Models](https://mlsanity.com/paper/2203.17260/) (2022)
- [Conditional Frechet Inception Distance](https://mlsanity.com/paper/2103.11521/) (2021)
- [Self-Diagnosing GAN: Diagnosing Underrepresented Samples in Generative Adversarial Networks](https://mlsanity.com/paper/2102.12033/) (2021)
- [Distilling and Transferring Knowledge via cGAN-generated Samples for Image Classification and Regression](https://mlsanity.com/paper/2104.03164/) (2021)
- [Model Extraction and Defenses on Generative Adversarial Networks](https://mlsanity.com/paper/2101.02069/) (2021)
- [Efficient Subsampling of Realistic Images From GANs Conditional on a Class or a Continuous Variable](https://mlsanity.com/paper/2103.11166/) (2021)
- [Robust Conformal Volume Estimation in 3D Medical Images](https://mlsanity.com/paper/2407.19938/) (2024)
- [Density Ratio Estimation and Neyman Pearson Classification with Missing Data](https://mlsanity.com/paper/2302.10655/) (2023)
- [Diffusion Stochastic Optimization for Min-Max Problems](https://mlsanity.com/paper/2401.14585/) (2024)
- [LapLoss: Laplacian Pyramid-based Multiscale loss for Image Translation](https://mlsanity.com/paper/2503.05974/) (2025)
- [Imbalance-Robust and Sampling-Efficient Continuous Conditional GANs via Adaptive Vicinal Learning and Auxiliary Regularization](https://mlsanity.com/paper/2508.01725/) (2025)

## Abstract (arXiv)

Filtering out unrealistic images from trained generative adversarial networks (GANs) has attracted considerable attention recently. Two density ratio based subsampling methods---Discriminator Rejection Sampling (DRS) and Metropolis-Hastings GAN (MH-GAN)---were recently proposed, and their effectiveness in improving GANs was demonstrated on multiple datasets. However, DRS and MH-GAN are based on discriminator based density ratio estimation (DRE) methods, so they may not work well if the discriminator in the trained GAN is far from optimal. Moreover, they do not apply to some GANs (e.g., MMD-GAN). In this paper, we propose a novel Softplus (SP) loss for DRE. Based on it, we develop a sample-based DRE method in a feature space learned by a specially designed and pre-trained ResNet-34 (DRE-F-SP). We derive the rate of convergence of a density ratio model trained under the SP loss. Then, we propose three different density ratio subsampling methods (DRE-F-SP+RS, DRE-F-SP+MH, and DRE-F-SP+SIR) for GANs based on DRE-F-SP. Our subsampling methods do not rely on the optimality of the discriminator and are suitable for all types of GANs. We empirically show our subsampling approach can substantially outperform DRS and MH-GAN on a synthetic dataset and the CIFAR-10 dataset, using multiple GANs.