Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics
Not on the map. Why?
Abstract (arXiv)
Since its emergence roughly a decade ago, microtask crowdsourcing has been attracting a heterogeneous set of workers from all over the globe. This paper sets out to explore the characteristics of the international crowd workforce and offers a cross-national comparison of crowdworker populations from ten countries. We provide an analysis and comparison of demographic characteristics and shed light on the significance of microtask income for workers situated in different national contexts. With over 11,000 individual responses, this study is the first large-scale country-level analysis of the characteristics of workers on the platform Appen (formerly CrowdFlower and Figure Eight), one of the two platforms dominating the microtask market. We find large differences between the characteristics of the crowd workforces of different countries, both regarding demography and regarding the importance of microtask income for workers. Furthermore, we find that the composition of the workforce in the ten countries was largely stable across samples taken at different points in time.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Dec 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 1; 2022: 3; 2023: 2; 2024: 2; 2025: 2.
10 papers.
- 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
- 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 67
- 2022CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotationcited by 59
- 2021Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotationcited by 51
- 2025ChatBench: From Static Benchmarks to Human-AI Evaluationcited by 31
- 2023Capturing Perspectives of Crowdsourced Annotators in Subjective Learning Taskscited by 28
- 2022Listening to Affected Communities to Define Extreme Speech: Dataset and Experimentscited by 21
- 2024Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and Governancecited by 3
- 2023Sociocultural knowledge is needed for selection of shots in hate speech detection taskscited by 1
- 2025Data Enrichment Work and AI Labor in Latin America and the Caribbeancited by 0
- 2025ChatBench: From Static Benchmarks to Human-AI Evaluation22 Mar 2025
- 2025Data Enrichment Work and AI Labor in Latin America and the Caribbean13 Jan 2025
- 2024Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and Governance26 Jul 2024
- 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models24 Apr 2024
- 2023Capturing Perspectives of Crowdsourced Annotators in Subjective Learning Tasks16 Nov 2023
- 2023Sociocultural knowledge is needed for selection of shots in hate speech detection tasks4 Apr 2023
- 2022CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation9 Jun 2022
- 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation1 May 2022
- 2022Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments22 Mar 2022
- 2021Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation8 Dec 2021