mlsanityData to 14 Oct 2026

A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk

arXiv 1712.05796 ↗First posted 14 Dec 2017cs.CY cs.HCcited by 39

Not on the map. Why?

Abstract (arXiv)

A growing number of people are working as part of on-line crowd work, which has been characterized by its low wages; yet, we know little about wage distribution and causes of low/high earnings. We recorded 2,676 workers performing 3.8 million tasks on Amazon Mechanical Turk. Our task-level analysis revealed that workers earned a median hourly wage of only ~2/h, and only 4% earned more than 7.25/h. The average requester pays more than 11/h, although lower-paying requesters post much more work. Our wage calculations are influenced by how unpaid work is included in our wage calculations, e.g., time spent searching for tasks, working on tasks that are rejected, and working on tasks that are ultimately not submitted. We further explore the characteristics of tasks and working patterns that yield higher hourly wages. Our analysis informs future platform design and worker tools to create a more positive future for crowd work.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 9; 2023: 4; 2024: 12; 2025: 5.

39 papers.

Sort:

By area:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  2. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 267
  3. 2023Large Language Models Enable Few-Shot Clusteringcited by 72
  4. 2022CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotationcited by 59
  5. 2022Identifying and Manipulating the Personality Traits of Language Modelscited by 40
  6. 2022Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithmcited by 35
  7. 2021CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Modelscited by 34
  8. 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
  9. 2021Does Putting a Linguist in the Loop Improve NLU Data Collection?cited by 29
  10. 2021FathomNet: A global image database for enabling artificial intelligence in the oceancited by 25
  11. 2022StyleFool: Fooling Video Classification Systems via Style Transfercited by 20
  12. 2023Incorporating Worker Perspectives into MTurk Annotation Practices for NLPcited by 18
27 more
  1. 2022The Data-Production Dispositifcited by 16
  2. 2021Investigating Memorization of Conspiracy Theories in Text Generationcited by 15
  3. 2021A Survey of NLP-Related Crowdsourcing HITs: what works and what does notcited by 15
  4. 2022Robots Enact Malignant Stereotypescited by 14
  5. 2024Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmarkcited by 14
  6. 2024CriticEval: Evaluating Large Language Model as Criticcited by 13
  7. 2024Evaluating LLM-based Personal Information Extraction and Countermeasurescited by 12
  8. 2021Quantifying and Avoiding Unfair Qualification Labour in Crowdsourcingcited by 11
  9. 2021Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?cited by 11
  10. 2024Multi-modal Retrieval Augmented Multi-modal Generation: Datasets, Evaluation Metrics and Strong Baselinescited by 9
  11. 2025Fairshare Data Pricing via Data Valuation for Large Language Modelscited by 8
  12. 2024The Great AI Witch Hunt: Reviewers Perception and (Mis)Conception of Generative AI in Research Writingcited by 6
  13. 2025HSCodeComp: A Realistic and Expert-level Benchmark for Deep Search Agents in Hierarchical Rule Applicationcited by 5
  14. 2021Characterizing Student Engagement Moods for Dropout Prediction in Question Pool Websitescited by 4
  15. 2024Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and Governancecited by 4
  16. 2024Stars, Stripes, and Silicon: Unravelling the ChatGPT's All-American, Monochrome, Cis-centric Biascited by 4
  17. 2023The ethical ambiguity of AI data enrichment: Measuring gaps in research ethics norms and practicescited by 3
  18. 2023Inclusive Portraits: Race-Aware Human-in-the-Loop Technologycited by 3
  19. 2024Towards Equitable Agile Research and Development of AI and Roboticscited by 2
  20. 2024Rethinking the Threat and Accessibility of Adversarial Attacks against Face Recognition Systemscited by 2
  21. 2025The Missing Reward: Active Inference in the Era of Experiencecited by 2
  22. 2022Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problemcited by 1
  23. 2024Temporal and Semantic Evaluation Metrics for Foundation Models in Post-Hoc Analysis of Robotic Sub-taskscited by 1
  24. 2022The "Collections as ML Data" Checklist for Machine Learning & Cultural Heritagecited by 0
  25. 2024A Culturally-Aware Tool for Crowdworkers: Leveraging Chronemics to Support Diverse Work Stylescited by 0
  26. 2025Data Enrichment Work and AI Labor in Latin America and the Caribbeancited by 0
  27. 2025Improving User Behavior Prediction: Leveraging Annotator Metadata in Supervised Machine Learning Modelscited by 0