# Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?

arXiv: https://arxiv.org/abs/2109.08131 · HTML: https://mlsanity.com/paper/2109.08131/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 100 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [Bringing the People Back In: Contesting Benchmark Machine Learning Datasets](https://mlsanity.com/paper/2007.07399/) (2020)
- [A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk](https://mlsanity.com/paper/1712.05796/) (2017)
- [Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision](https://mlsanity.com/paper/2007.14886/) (2020)
- [Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media](https://mlsanity.com/paper/1704.01347/) (2017)
- [Data Science as Political Action: Grounding Data Science in a Politics of Justice](https://mlsanity.com/paper/1811.03435/) (2018)
- [Data Vision: Learning to See Through Algorithmic Abstraction](https://mlsanity.com/paper/2002.03387/) (2020)
- [Measuring Social Biases of Crowd Workers using Counterfactual Queries](https://mlsanity.com/paper/2004.02028/) (2020)
- [AI for the Common Good?! Pitfalls, challenges, and Ethics Pen-Testing](https://mlsanity.com/paper/1810.12847/) (2018)
- [The Future of Work Is Here: Toward a Comprehensive Approach to Artificial Intelligence and Labour](https://mlsanity.com/paper/2007.05843/) (2020)
- [Wisdom for the Crowd: Discoursive Power in Annotation Instructions for Computer Vision](https://mlsanity.com/paper/2105.10990/) (2021)

## Built on by (most cited first, 11 of 11)

- [Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits](https://mlsanity.com/paper/2205.06922/) (2022)
- [The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing](https://mlsanity.com/paper/2407.07786/) (2024)
- [In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review](https://mlsanity.com/paper/2501.10727/) (2025)
- [Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use](https://mlsanity.com/paper/2405.12458/) (2024)
- [The Ethics of AI Value Chains](https://mlsanity.com/paper/2307.16787/) (2023)
- [Making Data Work Count](https://mlsanity.com/paper/2311.18046/) (2023)
- [The Cadaver in the Machine: The Social Practices of Measurement and Validation in Motion Capture Technology](https://mlsanity.com/paper/2401.10877/) (2024)
- [Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Power](https://mlsanity.com/paper/2604.21043/) (2026)
- [Gender Bias Detection in Court Decisions: A Brazilian Case Study](https://mlsanity.com/paper/2406.00393/) (2024)
- [Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation](https://mlsanity.com/paper/2602.18483/) (2026)
- [Relative Principals, Pluralistic Alignment, and the Structural Value Alignment Problem](https://mlsanity.com/paper/2604.20805/) (2026)

## Abstract (arXiv)

Research in machine learning (ML) has primarily argued that models trained on incomplete or biased datasets can lead to discriminatory outputs. In this commentary, we propose moving the research focus beyond bias-oriented framings by adopting a power-aware perspective to "study up" ML datasets. This means accounting for historical inequities, labor conditions, and epistemological standpoints inscribed in data. We draw on HCI and CSCW work to support our argument, critically analyze previous research, and point at two co-existing lines of work within our community -- one bias-oriented, the other power-aware. This way, we highlight the need for dialogue and cooperation in three areas: data quality, data work, and data documentation. In the first area, we argue that reducing societal problems to "bias" misses the context-based nature of data. In the second one, we highlight the corporate forces and market imperatives involved in the labor of data workers that subsequently shape ML datasets. Finally, we propose expanding current transparency-oriented efforts in dataset documentation to reflect the social contexts of data design and production.