# A Survey on Task Assignment in Crowdsourcing

arXiv: https://arxiv.org/abs/2111.08501 · HTML: https://mlsanity.com/paper/2111.08501/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2021.

## Built on by (most cited first, 8 of 8)

- [Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent](https://mlsanity.com/paper/2502.11267/) (2025)
- [What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns](https://mlsanity.com/paper/2504.15815/) (2025)
- [Towards AI-Empowered Crowdsourcing](https://mlsanity.com/paper/2212.14676/) (2022)
- [Efficient Online Crowdsourcing with Complex Annotations](https://mlsanity.com/paper/2401.15116/) (2024)
- [A Dataset for the Validation of Truth Inference Algorithms Suitable for Online Deployment](https://mlsanity.com/paper/2403.08826/) (2024)
- [DeCo: Defect-Aware Modeling with Contrasting Matching for Optimizing Task Assignment in Online IC Testing](https://mlsanity.com/paper/2505.00278/) (2025)
- [A Multi-Agent Reinforcement Learning Approach for Cooperative Air-Ground-Human Crowdsensing in Emergency Rescue](https://mlsanity.com/paper/2505.06997/) (2025)
- [A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Tasks](https://mlsanity.com/paper/2510.11897/) (2025)

## Abstract (arXiv)

Quality improvement methods are essential to gathering high-quality crowdsourced data, both for research and industry applications. A popular and broadly applicable method is task assignment that dynamically adjusts crowd workflow parameters. In this survey, we review task assignment methods that address: heterogeneous task assignment, question assignment, and plurality problems in crowdsourcing. We discuss and contrast how these methods estimate worker performance, and highlight potential challenges in their implementation. Finally, we discuss future research directions for task assignment methods, and how crowdsourcing platforms and other stakeholders can benefit from them.