# Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision

arXiv: https://arxiv.org/abs/2007.14886 · HTML: https://mlsanity.com/paper/2007.14886/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2020.

## Built on by (most cited first, 25 of 25)

- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [A Survey on Bias in Visual Datasets](https://mlsanity.com/paper/2107.07919/) (2021)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation](https://mlsanity.com/paper/2206.08931/) (2022)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithm](https://mlsanity.com/paper/2207.02942/) (2022)
- [Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits](https://mlsanity.com/paper/2205.06922/) (2022)
- [Participation in the age of foundation models](https://mlsanity.com/paper/2405.19479/) (2024)
- [Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desiderata](https://mlsanity.com/paper/2206.02923/) (2022)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Disentangling Perceptions of Offensiveness: Cultural and Moral Correlates](https://mlsanity.com/paper/2312.06861/) (2023)
- [Plurals: A System for Guiding LLMs Via Simulated Social Ensembles](https://mlsanity.com/paper/2409.17213/) (2024)
- [The Data-Production Dispositif](https://mlsanity.com/paper/2205.11963/) (2022)
- [Computer Vision and Conflicting Values: Describing People with Automated Alt Text](https://mlsanity.com/paper/2105.12754/) (2021)
- [Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?](https://mlsanity.com/paper/2109.08131/) (2021)
- [Ground Truth Or Dare: Factors Affecting The Creation Of Medical Datasets For Training AI](https://mlsanity.com/paper/2309.12327/) (2023)
- [Subverting machines, fluctuating identities: Re-learning human categorization](https://mlsanity.com/paper/2205.13740/) (2022)
- [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [A Survey on Image-text Multimodal Models](https://mlsanity.com/paper/2309.15857/) (2023)
- [Wisdom for the Crowd: Discoursive Power in Annotation Instructions for Computer Vision](https://mlsanity.com/paper/2105.10990/) (2021)
- [Unraveling the Interconnected Axes of Heterogeneity in Machine Learning for Democratic and Inclusive Advancements](https://mlsanity.com/paper/2306.10043/) (2023)
- [Making Data Work Count](https://mlsanity.com/paper/2311.18046/) (2023)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation](https://mlsanity.com/paper/2602.18483/) (2026)

## Abstract (arXiv)

The interpretation of data is fundamental to machine learning. This paper investigates practices of image data annotation as performed in industrial contexts. We define data annotation as a sense-making practice, where annotators assign meaning to data through the use of labels. Previous human-centered investigations have largely focused on annotators subjectivity as a major cause for biased labels. We propose a wider view on this issue: guided by constructivist grounded theory, we conducted several weeks of fieldwork at two annotation companies. We analyzed which structures, power relations, and naturalized impositions shape the interpretation of data. Our results show that the work of annotators is profoundly informed by the interests, values, and priorities of other actors above their station. Arbitrary classifications are vertically imposed on annotators, and through them, on data. This imposition is largely naturalized. Assigning meaning to data is often presented as a technical matter. This paper shows it is, in fact, an exercise of power with multiple implications for individuals and society.