# MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining

arXiv: https://arxiv.org/abs/2110.08009 · HTML: https://mlsanity.com/paper/2110.08009/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 71 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 30 of 46)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [Unrolled Generative Adversarial Networks](https://mlsanity.com/paper/1611.02163/) (2016)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [Energy-based Generative Adversarial Network](https://mlsanity.com/paper/1609.03126/) (2016)
- [Diversity-Sensitive Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1901.09024/) (2019)
- [Sylvester Normalizing Flows for Variational Inference](https://mlsanity.com/paper/1803.05649/) (2018)
- [Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling](https://mlsanity.com/paper/2003.06060/) (2020)
- [Spherical Latent Spaces for Stable Variational Autoencoders](https://mlsanity.com/paper/1808.10805/) (2018)
- [Generative Multi-Adversarial Networks](https://mlsanity.com/paper/1611.01673/) (2016)
- [Mad Max: Affine Spline Insights into Deep Learning](https://mlsanity.com/paper/1805.06576/) (2018)

## Built on by (most cited first, 23 of 23)

- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [Balancing Act: Distribution-Guided Debiasing in Diffusion Models](https://mlsanity.com/paper/2402.18206/) (2024)
- [SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundaries](https://mlsanity.com/paper/2302.12828/) (2023)
- [Diagnosing and Fixing Manifold Overfitting in Deep Generative Models](https://mlsanity.com/paper/2204.07172/) (2022)
- [Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models](https://mlsanity.com/paper/2209.06970/) (2022)
- [Fair Generative Models via Transfer Learning](https://mlsanity.com/paper/2212.00926/) (2022)
- [Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Values](https://mlsanity.com/paper/2203.01993/) (2022)
- [Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation](https://mlsanity.com/paper/2312.01648/) (2023)
- [On Measuring Fairness in Generative Models](https://mlsanity.com/paper/2310.19297/) (2023)
- [What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models](https://mlsanity.com/paper/2408.08307/) (2024)
- [FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation](https://mlsanity.com/paper/2410.18615/) (2024)
- [Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling](https://mlsanity.com/paper/2212.02090/) (2022)
- [Gaussian Harmony: Attaining Fairness in Diffusion-based Face Generation Models](https://mlsanity.com/paper/2312.14976/) (2023)
- [Training Dynamics of Deep Network Linear Regions](https://mlsanity.com/paper/2310.12977/) (2023)
- [Benchmarking the Fairness of Image Upsampling Methods](https://mlsanity.com/paper/2401.13555/) (2024)
- [Controlling the Fidelity and Diversity of Deep Generative Models via Pseudo Density](https://mlsanity.com/paper/2407.08659/) (2024)
- [PFGuard: A Generative Framework with Privacy and Fairness Safeguards](https://mlsanity.com/paper/2410.02246/) (2024)
- [Diverse Rare Sample Generation with Pretrained GANs](https://mlsanity.com/paper/2412.19543/) (2024)
- [Spider GAN: Leveraging Friendly Neighbors to Accelerate GAN Training](https://mlsanity.com/paper/2305.07613/) (2023)
- [Mitigating over-exploration in latent space optimization using LES](https://mlsanity.com/paper/2406.09657/) (2024)
- [Curvature Enhanced Data Augmentation for Regression](https://mlsanity.com/paper/2506.06853/) (2025)
- [Equalized Generative Treatment: Matching f-divergences for Fairness in Generative Models](https://mlsanity.com/paper/2602.08660/) (2026)
- [On the Geometry of Deep Learning](https://mlsanity.com/paper/2408.04809/) (2024)

## Abstract (arXiv)

Deep Generative Networks (DGNs) are extensively employed in Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and their variants to approximate the data manifold and distribution. However, training samples are often distributed in a non-uniform fashion on the manifold, due to costs or convenience of collection. For example, the CelebA dataset contains a large fraction of smiling faces. These inconsistencies will be reproduced when sampling from the trained DGN, which is not always preferred, e.g., for fairness or data augmentation. In response, we develop MaGNET, a novel and theoretically motivated latent space sampler for any pre-trained DGN, that produces samples uniformly distributed on the learned manifold. We perform a range of experiments on various datasets and DGNs, e.g., for the state-of-the-art StyleGAN2 trained on FFHQ dataset, uniform sampling via MaGNET increases distribution precision and recall by 4.1% & 3.0% and decreases gender bias by 41.2%, without requiring labels or retraining. As uniform distribution does not imply uniform semantic distribution, we also explore separately how semantic attributes of generated samples vary under MaGNET sampling.