# Characterizing Bias in Classifiers using Generative Models

arXiv: https://arxiv.org/abs/1906.11891 · HTML: https://mlsanity.com/paper/1906.11891/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 23 of 23)

- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation](https://mlsanity.com/paper/2404.01030/) (2024)
- [Camera Measurement of Physiological Vital Signs](https://mlsanity.com/paper/2111.11547/) (2021)
- [Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data](https://mlsanity.com/paper/2310.16524/) (2023)
- [Identifying and Improving Disability Bias in GPT-Based Resume Screening](https://mlsanity.com/paper/2402.01732/) (2024)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [Mitigating Dataset Bias by Using Per-sample Gradient](https://mlsanity.com/paper/2205.15704/) (2022)
- [FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations](https://mlsanity.com/paper/2202.06240/) (2022)
- [Anatomizing Bias in Facial Analysis](https://mlsanity.com/paper/2112.06522/) (2021)
- [Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement](https://mlsanity.com/paper/2303.12059/) (2023)
- [Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasets](https://mlsanity.com/paper/2305.09072/) (2023)
- [Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing](https://mlsanity.com/paper/2110.04902/) (2021)
- [Uncovering Bias in Face Generation Models](https://mlsanity.com/paper/2302.11562/) (2023)
- [Synthetic Data in Healthcare](https://mlsanity.com/paper/2304.03243/) (2023)
- [ET-AL: Entropy-Targeted Active Learning for Bias Mitigation in Materials Data](https://mlsanity.com/paper/2211.07881/) (2022)
- [Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolation](https://mlsanity.com/paper/2307.12219/) (2023)
- [Quality-Diversity Generative Sampling for Learning with Synthetic Data](https://mlsanity.com/paper/2312.14369/) (2023)
- [Enhancing Fairness of Visual Attribute Predictors](https://mlsanity.com/paper/2207.05727/) (2022)
- [Evaluation of Human and Machine Face Detection using a Novel Distinctive Human Appearance Dataset](https://mlsanity.com/paper/2111.00660/) (2021)
- [On Biased Behavior of GANs for Face Verification](https://mlsanity.com/paper/2208.13061/) (2022)
- [Denoising after Entropy-based Debiasing A Robust Training Method for Dataset Bias with Noisy Labels](https://mlsanity.com/paper/2212.01189/) (2022)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Auditing Facial Emotion Recognition Datasets for Posed Expressions and Racial Bias](https://mlsanity.com/paper/2507.10755/) (2025)

## Abstract (arXiv)

Models that are learned from real-world data are often biased because the data used to train them is biased. This can propagate systemic human biases that exist and ultimately lead to inequitable treatment of people, especially minorities. To characterize bias in learned classifiers, existing approaches rely on human oracles labeling real-world examples to identify the "blind spots" of the classifiers; these are ultimately limited due to the human labor required and the finite nature of existing image examples. We propose a simulation-based approach for interrogating classifiers using generative adversarial models in a systematic manner. We incorporate a progressive conditional generative model for synthesizing photo-realistic facial images and Bayesian Optimization for an efficient interrogation of independent facial image classification systems. We show how this approach can be used to efficiently characterize racial and gender biases in commercial systems.