# A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk

arXiv: https://arxiv.org/abs/1712.05796 · HTML: https://mlsanity.com/paper/1712.05796/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2017.

## Built on by (most cited first, 30 of 39)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning](https://mlsanity.com/paper/2110.13214/) (2021)
- [Large Language Models Enable Few-Shot Clustering](https://mlsanity.com/paper/2307.00524/) (2023)
- [CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation](https://mlsanity.com/paper/2206.08931/) (2022)
- [Identifying and Manipulating the Personality Traits of Language Models](https://mlsanity.com/paper/2212.10276/) (2022)
- [Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithm](https://mlsanity.com/paper/2207.02942/) (2022)
- [CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models](https://mlsanity.com/paper/2112.11941/) (2021)
- [Gradient based Feature Attribution in Explainable AI: A Technical Review](https://mlsanity.com/paper/2403.10415/) (2024)
- [Does Putting a Linguist in the Loop Improve NLU Data Collection?](https://mlsanity.com/paper/2104.07179/) (2021)
- [FathomNet: A global image database for enabling artificial intelligence in the ocean](https://mlsanity.com/paper/2109.14646/) (2021)
- [StyleFool: Fooling Video Classification Systems via Style Transfer](https://mlsanity.com/paper/2203.16000/) (2022)
- [Incorporating Worker Perspectives into MTurk Annotation Practices for NLP](https://mlsanity.com/paper/2311.02802/) (2023)
- [The Data-Production Dispositif](https://mlsanity.com/paper/2205.11963/) (2022)
- [Investigating Memorization of Conspiracy Theories in Text Generation](https://mlsanity.com/paper/2101.00379/) (2021)
- [A Survey of NLP-Related Crowdsourcing HITs: what works and what does not](https://mlsanity.com/paper/2111.05241/) (2021)
- [Robots Enact Malignant Stereotypes](https://mlsanity.com/paper/2207.11569/) (2022)
- [Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmark](https://mlsanity.com/paper/2411.15488/) (2024)
- [CriticEval: Evaluating Large Language Model as Critic](https://mlsanity.com/paper/2402.13764/) (2024)
- [Evaluating LLM-based Personal Information Extraction and Countermeasures](https://mlsanity.com/paper/2408.07291/) (2024)
- [Quantifying and Avoiding Unfair Qualification Labour in Crowdsourcing](https://mlsanity.com/paper/2105.12762/) (2021)
- [Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?](https://mlsanity.com/paper/2109.08131/) (2021)
- [Multi-modal Retrieval Augmented Multi-modal Generation: Datasets, Evaluation Metrics and Strong Baselines](https://mlsanity.com/paper/2411.16365/) (2024)
- [Fairshare Data Pricing via Data Valuation for Large Language Models](https://mlsanity.com/paper/2502.00198/) (2025)
- [The Great AI Witch Hunt: Reviewers Perception and (Mis)Conception of Generative AI in Research Writing](https://mlsanity.com/paper/2407.12015/) (2024)
- [HSCodeComp: A Realistic and Expert-level Benchmark for Deep Search Agents in Hierarchical Rule Application](https://mlsanity.com/paper/2510.19631/) (2025)
- [Characterizing Student Engagement Moods for Dropout Prediction in Question Pool Websites](https://mlsanity.com/paper/2102.00423/) (2021)
- [Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and Governance](https://mlsanity.com/paper/2408.01458/) (2024)
- [Stars, Stripes, and Silicon: Unravelling the ChatGPT's All-American, Monochrome, Cis-centric Bias](https://mlsanity.com/paper/2410.13868/) (2024)
- [The ethical ambiguity of AI data enrichment: Measuring gaps in research ethics norms and practices](https://mlsanity.com/paper/2306.01800/) (2023)
- [Inclusive Portraits: Race-Aware Human-in-the-Loop Technology](https://mlsanity.com/paper/2311.03567/) (2023)

## Abstract (arXiv)

A growing number of people are working as part of on-line crowd work, which has been characterized by its low wages; yet, we know little about wage distribution and causes of low/high earnings. We recorded 2,676 workers performing 3.8 million tasks on Amazon Mechanical Turk. Our task-level analysis revealed that workers earned a median hourly wage of only ~2/h, and only 4% earned more than 7.25/h. The average requester pays more than 11/h, although lower-paying requesters post much more work. Our wage calculations are influenced by how unpaid work is included in our wage calculations, e.g., time spent searching for tasks, working on tasks that are rejected, and working on tasks that are ultimately not submitted. We further explore the characteristics of tasks and working patterns that yield higher hourly wages. Our analysis informs future platform design and worker tools to create a more positive future for crowd work.