# Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics

arXiv: https://arxiv.org/abs/2012.02394 · HTML: https://mlsanity.com/paper/2012.02394/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches](https://mlsanity.com/paper/2205.04460/) (2022)
- [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions](https://mlsanity.com/paper/2107.09546/) (2021)
- [China and the U.S. produce more impactful AI research when collaborating together](https://mlsanity.com/paper/2304.11123/) (2023)
- [Ethics in the Age of AI: An Analysis of AI Practitioners' Awareness and Challenges](https://mlsanity.com/paper/2307.10057/) (2023)
- [Standard Language Ideology in AI-Generated Language](https://mlsanity.com/paper/2406.08726/) (2024)
- [Agent Bain vs. Agent McKinsey: A New Text-to-SQL Benchmark for the Business Domain](https://mlsanity.com/paper/2510.07309/) (2025)
- [On Optimal Steering to Achieve Exact Fairness](https://mlsanity.com/paper/2509.15759/) (2025)
- [LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals](https://mlsanity.com/paper/2601.10700/) (2026)
- [Uncovering the Source of Machine Bias](https://mlsanity.com/paper/2201.03092/) (2022)
- [Introducing Construct Theory as a Standard Methodology for Inclusive AI Models](https://mlsanity.com/paper/2304.09867/) (2023)

## Abstract (arXiv)

Why do biased predictions arise? What interventions can prevent them? We evaluate 8.2 million algorithmic predictions of math performance from ≈400 AI engineers, each of whom developed an algorithm under a randomly assigned experimental condition. Our treatment arms modified programmers' incentives, training data, awareness, and/or technical knowledge of AI ethics. We then assess out-of-sample predictions from their algorithms using randomized audit manipulations of algorithm inputs and ground-truth math performance for 20K subjects. We find that biased predictions are mostly caused by biased training data. However, one-third of the benefit of better training data comes through a novel economic mechanism: Engineers exert greater effort and are more responsive to incentives when given better training data. We also assess how performance varies with programmers' demographic characteristics, and their performance on a psychological test of implicit bias (IAT) concerning gender and careers. We find no evidence that female, minority and low-IAT engineers exhibit lower bias or discrimination in their code. However, we do find that prediction errors are correlated within demographic groups, which creates performance improvements through cross-demographic averaging. Finally, we quantify the benefits and tradeoffs of practical managerial or policy interventions such as technical advice, simple reminders, and improved incentives for decreasing algorithmic bias.