# Learning to Incentivize: Eliciting Effort via Output Agreement

arXiv: https://arxiv.org/abs/1604.04928 · HTML: https://mlsanity.com/paper/1604.04928/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2016.

## Built on by (most cited first, 8 of 8)

- [Optimal and Differentially Private Data Acquisition: Central and Local Mechanisms](https://mlsanity.com/paper/2201.03968/) (2022)
- [The Importance of Human-Labeled Data in the Era of LLMs](https://mlsanity.com/paper/2306.14910/) (2023)
- [Learnersourcing in the Age of AI: Student, Educator and Machine Partnerships for Content Creation](https://mlsanity.com/paper/2306.06386/) (2023)
- [SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing](https://mlsanity.com/paper/2509.11265/) (2025)
- [Truthful Incentive Mechanism for Federated Learning with Crowdsourced Data Labeling](https://mlsanity.com/paper/2302.00106/) (2023)
- [Auctions and Peer Prediction for Academic Peer Review](https://mlsanity.com/paper/2109.00923/) (2021)
- [Incentivizing an Unknown Crowd](https://mlsanity.com/paper/2109.04226/) (2021)
- [Eliciting Truthful Reports with Partial Signals in Repeated Games](https://mlsanity.com/paper/2109.04343/) (2021)

## Abstract (arXiv)

In crowdsourcing when there is a lack of verification for contributed answers, output agreement mechanisms are often used to incentivize participants to provide truthful answers when the correct answer is hold by the majority. In this paper, we focus on using output agreement mechanisms to elicit effort, in addition to eliciting truthful answers, from a population of workers. We consider a setting where workers have heterogeneous cost of effort exertion and examine the data requester's problem of deciding the reward level in output agreement for optimal elicitation. In particular, when the requester knows the cost distribution, we derive the optimal reward level for output agreement mechanisms. This is achieved by first characterizing Bayesian Nash equilibria of output agreement mechanisms for a given reward level. When the requester does not know the cost distribution, we develop sequential mechanisms that combine learning the cost distribution with incentivizing effort exertion to approximately determine the optimal reward level.