# A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?

arXiv: https://arxiv.org/abs/2302.07159 · HTML: https://mlsanity.com/paper/2302.07159/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications](https://mlsanity.com/paper/2108.02818/) (2021)
- [A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning](https://mlsanity.com/paper/2203.11933/) (2022)
- [How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?](https://mlsanity.com/paper/2210.15230/) (2022)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels](https://mlsanity.com/paper/1512.06974/) (2015)
- [American == White in Multimodal Language-and-Image AI](https://mlsanity.com/paper/2207.00691/) (2022)
- [Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models](https://mlsanity.com/paper/2206.11684/) (2022)
- [Underspecification in Scene Description-to-Depiction Tasks](https://mlsanity.com/paper/2210.05815/) (2022)
- [Diversity and Inclusion Metrics in Subset Selection](https://mlsanity.com/paper/2002.03256/) (2020)
- [Stereotyping and Bias in the Flickr30K Dataset](https://mlsanity.com/paper/1605.06083/) (2016)
- [Computer Vision and Conflicting Values: Describing People with Automated Alt Text](https://mlsanity.com/paper/2105.12754/) (2021)
- [Assessing Dataset Bias in Computer Vision](https://mlsanity.com/paper/2205.01811/) (2022)

## Built on by (most cited first, 25 of 25)

- [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023)
- [Editing Implicit Assumptions in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.08084/) (2023)
- [The Bias Amplification Paradox in Text-to-Image Generation](https://mlsanity.com/paper/2308.00755/) (2023)
- [Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation](https://mlsanity.com/paper/2404.01030/) (2024)
- [UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Images](https://mlsanity.com/paper/2405.03486/) (2024)
- [Inspecting the Geographical Representativeness of Images from Text-to-Image Models](https://mlsanity.com/paper/2305.11080/) (2023)
- ['Person' == Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion](https://mlsanity.com/paper/2310.19981/) (2023)
- [ReFACT: Updating Text-to-Image Models by Editing the Text Encoder](https://mlsanity.com/paper/2306.00738/) (2023)
- [Fair Text-to-Image Diffusion via Fair Mapping](https://mlsanity.com/paper/2311.17695/) (2023)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [Mapping the Ethics of Generative AI: A Comprehensive Scoping Review](https://mlsanity.com/paper/2402.08323/) (2024)
- [BiasDora: Exploring Hidden Biased Associations in Vision-Language Models](https://mlsanity.com/paper/2407.02066/) (2024)
- [A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models](https://mlsanity.com/paper/2402.13636/) (2024)
- [Social Perception of Faces in a Vision-Language Model](https://mlsanity.com/paper/2408.14435/) (2024)
- [Analysing Gender Bias in Text-to-Image Models using Object Detection](https://mlsanity.com/paper/2307.08025/) (2023)
- [Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolution](https://mlsanity.com/paper/2408.17285/) (2024)
- [The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects](https://mlsanity.com/paper/2402.11089/) (2024)
- [Bias Amplification in Stable Diffusion's Representation of Stigma Through Skin Tones and Their Homogeneity](https://mlsanity.com/paper/2508.17465/) (2025)
- [Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?](https://mlsanity.com/paper/2501.15775/) (2025)
- [Multi-Group Proportional Representation for Text-to-Image Models](https://mlsanity.com/paper/2505.24023/) (2025)
- [Explainable Search and Discovery of Visual Cultural Heritage Collections with Multimodal Large Language Models](https://mlsanity.com/paper/2411.04663/) (2024)
- [Implicit Priors Editing in Stable Diffusion via Targeted Token Adjustment](https://mlsanity.com/paper/2412.03400/) (2024)
- [The Digital Mirror: Gender Bias and Occupational Stereotypes in AI-Generated Images](https://mlsanity.com/paper/2510.08628/) (2025)
- [Gender Artifacts from Art History to Text-to-Image Generation](https://mlsanity.com/paper/2606.05829/) (2026)
- [IMPLICIT-Bench: Measuring Implicit Bias in Text-to-Image Models under Neutral Prompts](https://mlsanity.com/paper/2609.24228/) (2026)

## Abstract (arXiv)

As text-to-image systems continue to grow in popularity with the general public, questions have arisen about bias and diversity in the generated images. Here, we investigate properties of images generated in response to prompts which are visually under-specified, but contain salient social attributes (e.g., 'a portrait of a threatening person' versus 'a portrait of a friendly person'). Grounding our work in social cognition theory, we find that in many cases, images contain similar demographic biases to those reported in the stereotype literature. However, trends are inconsistent across different models and further investigation is warranted.