mlsanityData to 14 Oct 2026

On Measuring Fairness in Generative Models

arXiv 2310.19297 ↗First posted 30 Oct 2023cs.LG cs.CV cs.CYcited by 8

Not on the map. Why?

Abstract (arXiv)

Recently, there has been increased interest in fair generative models. In this work, we conduct, for the first time, an in-depth study on fairness measurement, a critical component in gauging progress on fair generative models. We make three contributions. First, we conduct a study that reveals that the existing fairness measurement framework has considerable measurement errors, even when highly accurate sensitive attribute (SA) classifiers are used. These findings cast doubts on previously reported fairness improvements. Second, to address this issue, we propose CLassifier Error-Aware Measurement (CLEAM), a new framework which uses a statistical model to account for inaccuracies in SA classifiers. Our proposed CLEAM reduces measurement errors significantly, e.g., 4.98% rightarrow 0.62% for StyleGAN2 w.r.t. Gender. Additionally, CLEAM achieves this with minimal additional overhead. Third, we utilize CLEAM to measure fairness in important text-to-image generator and GANs, revealing considerable biases in these models that raise concerns about their applications. Code and more resources: https://sutd-visual-computing-group.github.io/CLEAM/.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 30 Oct 2023.

Builds on

47 of 67 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2017On Calibration of Modern Neural Networkscited by 4,960
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  11. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
35 more
  1. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  2. 2014Conditional Generative Adversarial Netscited by 2,162
  3. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  4. 2022Visual Prompt Tuningcited by 1,681
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  6. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  7. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  8. 2014Certifying and removing disparate impactcited by 715
  9. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  10. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  11. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 565
  12. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 366
  13. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  14. 2018GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classificationcited by 242
  15. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
  16. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  17. 2018Fair Clustering Through Fairletscited by 176
  18. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  19. 2021Generative Adversarial Transformerscited by 120
  20. 2019Fair Generative Modeling via Weak Supervisioncited by 114
  21. 201850 Years of Test (Un)fairness: Lessons for Machine Learningcited by 101
  22. 2020Improving the Fairness of Deep Generative Models without Retrainingcited by 46
  23. 2020Inclusive GAN: Improving Data and Minority Coverage in Generative Modelscited by 44
  24. 2018Fair and Diverse DPP-based Data Summarizationcited by 39
  25. 2022Few-shot Image Generation via Adaptation-Aware Kernel Modulationcited by 33
  26. 2021Measuring Fairness in Generative Modelscited by 31
  27. 2022Studying Bias in GANs through the Lens of Racecited by 31
  28. 2018Robustness of Conditional GANs to Noisy Labelscited by 27
  29. 2022Fair Generative Models via Transfer Learningcited by 27
  30. 2022Discovering Transferable Forensic Features for CNN-generated Images Detectioncited by 24
  31. 2021MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retrainingcited by 23
  32. 2021Revisit Multimodal Meta-Learning through the Lens of Multi-Task Learningcited by 22
  33. 2023Exploring Incompatible Knowledge Transfer in Few-shot Image Generationcited by 19
  34. 2023A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shotcited by 19
  35. 2021Auditing for Diversity using Representative Examplescited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 3.

8 papers.

  1. 2023Stable Diffusion Exposed: Gender Bias from Prompt to Imagecited by 25
  2. 2024Fairness and Bias in Multimodal AI: A Surveycited by 20
  3. 2025Debiasing Diffusion Model: Enhancing Fairness through Latent Representation Learning in Stable Diffusion Modelcited by 5
  4. 2025Identities are not Interchangeable: The Problem of Overgeneralization in Fair Machine Learningcited by 5
  5. 2024On the Fairness, Diversity and Reliability of Text-to-Image Generative Modelscited by 2
  6. 2024AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generationcited by 1
  7. 2024MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Biascited by 1
  8. 2025Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Modelscited by 1