# Computer Vision and Conflicting Values: Describing People with Automated Alt Text

arXiv: https://arxiv.org/abs/2105.12754 · HTML: https://mlsanity.com/paper/2105.12754/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 69 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 13 of 13)

- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)
- [From Captions to Visual Concepts and Back](https://mlsanity.com/paper/1411.4952/) (2014)
- [Racial categories in machine learning](https://mlsanity.com/paper/1811.11668/) (2018)
- [Stereotyping and Bias in the Flickr30K Dataset](https://mlsanity.com/paper/1605.06083/) (2016)
- [Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision](https://mlsanity.com/paper/2007.14886/) (2020)
- [An Ethical Highlighter for People-Centric Dataset Creation](https://mlsanity.com/paper/2011.13583/) (2020)
- [Rich Image Captioning in the Wild](https://mlsanity.com/paper/1603.09016/) (2016)
- ["Is there anything else I can help you with?": Challenges in Deploying an On-Demand Crowd-Powered Conversational Agent](https://mlsanity.com/paper/1708.03044/) (2017)
- [What is the Point of Fairness? Disability, AI and The Complexity of Justice](https://mlsanity.com/paper/1908.01024/) (2019)
- [The Effect of Computer-Generated Descriptions on Photo-Sharing Experiences of People with Visual Impairments](https://mlsanity.com/paper/1805.01515/) (2018)

## Built on by (most cited first, 14 of 14)

- [VisText: A Benchmark for Semantically Rich Chart Captioning](https://mlsanity.com/paper/2307.05356/) (2023)
- [Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images](https://mlsanity.com/paper/2402.05779/) (2024)
- [Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content](https://mlsanity.com/paper/2110.04406/) (2021)
- [Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals](https://mlsanity.com/paper/2405.20152/) (2024)
- [A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?](https://mlsanity.com/paper/2302.07159/) (2023)
- [Consensus and Subjectivity of Skin Tone Annotation for ML Fairness](https://mlsanity.com/paper/2305.09073/) (2023)
- [Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Models](https://mlsanity.com/paper/2502.16098/) (2025)
- [Ground Truth Or Dare: Factors Affecting The Creation Of Medical Datasets For Training AI](https://mlsanity.com/paper/2309.12327/) (2023)
- [AltGen: AI-Driven Alt Text Generation for Enhancing EPUB Accessibility](https://mlsanity.com/paper/2501.00113/) (2024)
- ["We are at the mercy of others' opinion": Supporting Blind People in Recreational Window Shopping with AI-infused Technology](https://mlsanity.com/paper/2405.06611/) (2024)
- [Alt-Text with Context: Improving Accessibility for Images on Twitter](https://mlsanity.com/paper/2305.14779/) (2023)
- [Not All Visitors are Bilingual: A Measurement Study of the Multilingual Web from an Accessibility Perspective](https://mlsanity.com/paper/2508.18328/) (2025)
- [Cripping AI: Reimagining AI Through Lived Disability Experiences](https://mlsanity.com/paper/2605.02080/) (2026)
- [How Humans and LLMs Read Gender into "Gender-Neutral" Physical Descriptions](https://mlsanity.com/paper/2609.16366/) (2026)

## Abstract (arXiv)

Scholars have recently drawn attention to a range of controversial issues posed by the use of computer vision for automatically generating descriptions of people in images. Despite these concerns, automated image description has become an important tool to ensure equitable access to information for blind and low vision people. In this paper, we investigate the ethical dilemmas faced by companies that have adopted the use of computer vision for producing alt text: textual descriptions of images for blind and low vision people, We use Facebook's automatic alt text tool as our primary case study. First, we analyze the policies that Facebook has adopted with respect to identity categories, such as race, gender, age, etc., and the company's decisions about whether to present these terms in alt text. We then describe an alternative -- and manual -- approach practiced in the museum community, focusing on how museums determine what to include in alt text descriptions of cultural artifacts. We compare these policies, using notable points of contrast to develop an analytic framework that characterizes the particular apprehensions behind these policy choices. We conclude by considering two strategies that seem to sidestep some of these concerns, finding that there are no easy ways to avoid the normative dilemmas posed by the use of computer vision to automate alt text.