# Multimodal Composite Association Score: Measuring Gender Bias in Generative Multimodal Models

arXiv: https://arxiv.org/abs/2304.13855 · HTML: https://mlsanity.com/paper/2304.13855/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 12 of 12)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases](https://mlsanity.com/paper/2010.15052/) (2020)
- [Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels](https://mlsanity.com/paper/1512.06974/) (2015)
- [InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometrics](https://mlsanity.com/paper/2004.06592/) (2020)
- [UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models](https://mlsanity.com/paper/2110.15499/) (2021)
- [A study on the distribution of social biases in self-supervised learning visual models](https://mlsanity.com/paper/2203.01854/) (2022)

## Built on by (most cited first, 21 of 21)

- [Capabilities of Gemini Models in Medicine](https://mlsanity.com/paper/2404.18416/) (2024)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation](https://mlsanity.com/paper/2404.01030/) (2024)
- ['Person' == Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion](https://mlsanity.com/paper/2310.19981/) (2023)
- [Stable Diffusion Exposed: Gender Bias from Prompt to Image](https://mlsanity.com/paper/2312.03027/) (2023)
- [Would Deep Generative Models Amplify Bias in Future Models?](https://mlsanity.com/paper/2404.03242/) (2024)
- [BiasDora: Exploring Hidden Biased Associations in Vision-Language Models](https://mlsanity.com/paper/2407.02066/) (2024)
- [Analysing Gender Bias in Text-to-Image Models using Object Detection](https://mlsanity.com/paper/2307.08025/) (2023)
- [Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributes](https://mlsanity.com/paper/2407.03623/) (2024)
- [Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions](https://mlsanity.com/paper/2505.20692/) (2025)
- [Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation](https://mlsanity.com/paper/2509.07596/) (2025)
- [Biased Attention: Do Vision Transformers Amplify Gender Bias More than Convolutional Neural Networks?](https://mlsanity.com/paper/2309.08760/) (2023)
- [AudRandAug: Random Image Augmentations for Audio Classification](https://mlsanity.com/paper/2309.04762/) (2023)
- [From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship](https://mlsanity.com/paper/2506.23101/) (2025)
- [More is Less? A Simulation-Based Approach to Dynamic Interactions between Biases in Multimodal Models](https://mlsanity.com/paper/2412.17505/) (2024)
- [Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations](https://mlsanity.com/paper/2509.04515/) (2025)
- [Gender Artifacts from Art History to Text-to-Image Generation](https://mlsanity.com/paper/2606.05829/) (2026)
- [FaceSaliencyAug: Mitigating Geographic, Gender and Stereotypical Biases via Saliency-Based Data Augmentation](https://mlsanity.com/paper/2410.14070/) (2024)
- [Saliency-Based diversity and fairness Metric and FaceKeepOriginalAugment: A Novel Approach for Enhancing Fairness and Diversity](https://mlsanity.com/paper/2411.00831/) (2024)
- [Investigating Disability Representations in Text-to-Image Models](https://mlsanity.com/paper/2602.04687/) (2026)
- [Newer Is Not Fairer: Gender Stereotyping in Text-to-Image AI Across Model Generations](https://mlsanity.com/paper/2609.18007/) (2026)

## Abstract (arXiv)

Generative multimodal models based on diffusion models have seen tremendous growth and advances in recent years. Models such as DALL-E and Stable Diffusion have become increasingly popular and successful at creating images from texts, often combining abstract ideas. However, like other deep learning models, they also reflect social biases they inherit from their training data, which is often crawled from the internet. Manually auditing models for biases can be very time and resource consuming and is further complicated by the unbounded and unconstrained nature of inputs these models can take. Research into bias measurement and quantification has generally focused on small single-stage models working on a single modality. Thus the emergence of multistage multimodal models requires a different approach. In this paper, we propose Multimodal Composite Association Score (MCAS) as a new method of measuring gender bias in multimodal generative models. Evaluating both DALL-E 2 and Stable Diffusion using this approach uncovered the presence of gendered associations of concepts embedded within the models. We propose MCAS as an accessible and scalable method of quantifying potential bias for models with different modalities and a range of potential biases.