# Improving Fairness and Mitigating MADness in Generative Models

arXiv: https://arxiv.org/abs/2405.13977 · HTML: https://mlsanity.com/paper/2405.13977/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 65 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 19 of 19)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [BayesFlow: Learning complex stochastic models with invertible neural networks](https://mlsanity.com/paper/2003.06281/) (2020)
- [Variational Autoencoder for Deep Learning of Images, Labels and Captions](https://mlsanity.com/paper/1609.08976/) (2016)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [Bias and Generalization in Deep Generative Models: An Empirical Study](https://mlsanity.com/paper/1811.03259/) (2018)
- [Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop](https://mlsanity.com/paper/2311.16822/) (2023)
- [On the Frequency Bias of Generative Models](https://mlsanity.com/paper/2111.02447/) (2021)
- [HyperShot: Few-Shot Learning by Kernel HyperNetworks](https://mlsanity.com/paper/2203.11378/) (2022)
- [Learning to Estimate Without Bias](https://mlsanity.com/paper/2110.12403/) (2021)
- [Bias Correction with Jackknife, Bootstrap, and Taylor Series](https://mlsanity.com/paper/1709.06183/) (2017)
- [Bias in parametric estimation: reduction and useful side-effects](https://mlsanity.com/paper/1311.6311/) (2013)

## Built on by (most cited first, 4 of 4)

- [Equalized Generative Treatment: Matching f-divergences for Fairness in Generative Models](https://mlsanity.com/paper/2602.08660/) (2026)
- [Hypernetwork-based approach for grid-independent functional data clustering](https://mlsanity.com/paper/2602.22823/) (2026)
- [A lift for input-convex neural net training](https://mlsanity.com/paper/2605.24274/) (2026)
- [SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data](https://mlsanity.com/paper/2609.12353/) (2026)

## Abstract (arXiv)

Generative models unfairly penalize data belonging to minority classes, suffer from model autophagy disorder (MADness), and learn biased estimates of the underlying distribution parameters. Our theoretical and empirical results show that training generative models with intentionally designed hypernetworks leads to models that 1) are more fair when generating datapoints belonging to minority classes 2) are more stable in a self-consumed (i.e., MAD) setting, and 3) learn parameters that are less statistically biased. To further mitigate unfairness, MADness, and bias, we introduce a regularization term that penalizes discrepancies between a generative model's estimated weights when trained on real data versus its own synthetic data. To facilitate training existing deep generative models within our framework, we offer a scalable implementation of hypernetworks that automatically generates a hypernetwork architecture for any given generative model.