# Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing

arXiv: https://arxiv.org/abs/1408.1387 · HTML: https://mlsanity.com/paper/1408.1387/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2014.

## Built on by (most cited first, 11 of 11)

- [Data Pricing in Machine Learning Pipelines](https://mlsanity.com/paper/2108.07915/) (2021)
- [The Importance of Human-Labeled Data in the Era of LLMs](https://mlsanity.com/paper/2306.14910/) (2023)
- [Truthful Incentive Mechanism for Federated Learning with Crowdsourced Data Labeling](https://mlsanity.com/paper/2302.00106/) (2023)
- [When Incentives Backfire, Data Stops Being Human](https://mlsanity.com/paper/2502.07732/) (2025)
- [More Data Can Lead Us Astray: Active Data Acquisition in the Presence of Label Bias](https://mlsanity.com/paper/2207.07723/) (2022)
- [Trading-Off Payments and Accuracy in Online Classification with Paid Stochastic Experts](https://mlsanity.com/paper/2307.00836/) (2023)
- [Learning From Crowdsourced Noisy Labels: A Signal Processing Perspective](https://mlsanity.com/paper/2407.06902/) (2024)
- [Beyond Statistical Estimation: Differentially Private Individual Computation via Shuffling](https://mlsanity.com/paper/2406.18145/) (2024)
- [Incentivizing an Unknown Crowd](https://mlsanity.com/paper/2109.04226/) (2021)
- [Incentivizing High-Quality Human Annotations with Golden Questions](https://mlsanity.com/paper/2505.19134/) (2025)
- [Truthful Online Preference Aggregation for LLM Fine-Tuning in Mobile Crowdsourcing](https://mlsanity.com/paper/2605.24052/) (2026)

## Abstract (arXiv)

Crowdsourcing has gained immense popularity in machine learning applications for obtaining large amounts of labeled data. Crowdsourcing is cheap and fast, but suffers from the problem of low-quality data. To address this fundamental challenge in crowdsourcing, we propose a simple payment mechanism to incentivize workers to answer only the questions that they are sure of and skip the rest. We show that surprisingly, under a mild and natural "no-free-lunch" requirement, this mechanism is the one and only incentive-compatible payment mechanism possible. We also show that among all possible incentive-compatible mechanisms (that may or may not satisfy no-free-lunch), our mechanism makes the smallest possible payment to spammers. We further extend our results to a more general setting in which workers are required to provide a quantized confidence for each question. Interestingly, this unique mechanism takes a "multiplicative" form. The simplicity of the mechanism is an added benefit. In preliminary experiments involving over 900 worker-task pairs, we observe a significant drop in the error rates under this unique mechanism for the same or lower monetary expenditure.