# Measuring Representational Harms in Image Captioning

arXiv: https://arxiv.org/abs/2206.07173 · HTML: https://mlsanity.com/paper/2206.07173/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 80 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 30 of 35)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](https://mlsanity.com/paper/2009.12991/) (2020)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning](https://mlsanity.com/paper/1612.01887/) (2016)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)
- [Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings](https://mlsanity.com/paper/1809.02169/) (2018)
- [Understanding and Evaluating Racial Biases in Image Captioning](https://mlsanity.com/paper/2106.08503/) (2021)
- [Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images](https://mlsanity.com/paper/1703.10660/) (2017)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Discriminability objective for training descriptive captions](https://mlsanity.com/paper/1803.04376/) (2018)
- [Towards Diverse and Natural Image Descriptions via a Conditional GAN](https://mlsanity.com/paper/1703.06029/) (2017)
- [Learning to Evaluate Image Captioning](https://mlsanity.com/paper/1806.06422/) (2018)
- [Convolutional Image Captioning](https://mlsanity.com/paper/1711.09151/) (2017)
- [Mitigating Gender Bias in Captioning Systems](https://mlsanity.com/paper/2006.08315/) (2020)
- [Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models](https://mlsanity.com/paper/1912.00578/) (2019)

## Built on by (most cited first, 30 of 36)

- [AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia](https://mlsanity.com/paper/2305.11844/) (2023)
- [Uncurated Image-Text Datasets: Shedding Light on Demographic Bias](https://mlsanity.com/paper/2304.02828/) (2023)
- [Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression](https://mlsanity.com/paper/2407.04965/) (2024)
- [Overwriting Pretrained Bias with Finetuning Data](https://mlsanity.com/paper/2303.06167/) (2023)
- [FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classification](https://mlsanity.com/paper/2207.10888/) (2022)
- [ToViLaG: Your Visual-Language Generative Model is Also An Evildoer](https://mlsanity.com/paper/2312.11523/) (2023)
- [Identifying Implicit Social Biases in Vision-Language Models](https://mlsanity.com/paper/2411.00997/) (2024)
- [SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generation](https://mlsanity.com/paper/2401.08053/) (2024)
- [Beyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation](https://mlsanity.com/paper/2402.01705/) (2024)
- [BendVLM: Test-Time Debiasing of Vision-Language Embeddings](https://mlsanity.com/paper/2411.04420/) (2024)
- [Safety and Fairness for Content Moderation in Generative Models](https://mlsanity.com/paper/2306.06135/) (2023)
- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [Harm Amplification in Text-to-Image Models](https://mlsanity.com/paper/2402.01787/) (2024)
- [Nichelle and Nancy: The Influence of Demographic Attributes and Tokenization Length on First Name Biases](https://mlsanity.com/paper/2305.16577/) (2023)
- [FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplication](https://mlsanity.com/paper/2404.16123/) (2024)
- [Fairness in Visual Clustering: A Novel Transformer Clustering Approach](https://mlsanity.com/paper/2304.07408/) (2023)
- [AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development](https://mlsanity.com/paper/2502.18682/) (2025)
- [VLSlice: Interactive Vision-and-Language Slice Discovery](https://mlsanity.com/paper/2309.06703/) (2023)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [Explaining CLIP's performance disparities on data from blind/low vision users](https://mlsanity.com/paper/2311.17315/) (2023)
- [Building Trustworthy Multimodal AI: A Review of Fairness, Transparency, and Ethics in Vision-Language Tasks](https://mlsanity.com/paper/2504.13199/) (2025)
- [Unveiling and Mitigating Bias in Large Language Model Recommendations: A Path to Fairness](https://mlsanity.com/paper/2409.10825/) (2024)
- [A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysis](https://mlsanity.com/paper/2210.04491/) (2022)
- [Evaluating Model Bias Requires Characterizing its Mistakes](https://mlsanity.com/paper/2407.10633/) (2024)
- [The Backfiring Effect of Weak AI Safety Regulation](https://mlsanity.com/paper/2503.20848/) (2025)
- [VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models](https://mlsanity.com/paper/2505.22897/) (2025)
- [Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution](https://mlsanity.com/paper/2508.07111/) (2025)
- [Are you sure? Measuring models bias in content moderation through uncertainty](https://mlsanity.com/paper/2509.22699/) (2025)
- [Do Image-Text Metrics Respect Semantic Invariances?](https://mlsanity.com/paper/2605.24702/) (2026)
- [Common-Sense Bias Modeling for Classification Tasks](https://mlsanity.com/paper/2401.13213/) (2024)

## Abstract (arXiv)

Previous work has largely considered the fairness of image captioning systems through the underspecified lens of "bias." In contrast, we present a set of techniques for measuring five types of representational harms, as well as the resulting measurements obtained for two of the most popular image captioning datasets using a state-of-the-art image captioning system. Our goal was not to audit this image captioning system, but rather to develop normatively grounded measurement techniques, in turn providing an opportunity to reflect on the many challenges involved. We propose multiple measurement techniques for each type of harm. We argue that by doing so, we are better able to capture the multi-faceted nature of each type of harm, in turn improving the (collective) validity of the resulting measurements. Throughout, we discuss the assumptions underlying our measurement approach and point out when they do not hold.