# Fundamental Limits of Budget-Fidelity Trade-off in Label Crowdsourcing

arXiv: https://arxiv.org/abs/1608.07328 · HTML: https://mlsanity.com/paper/1608.07328/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2016.

## Built on by (most cited first, 1 of 1)

- [How to Query An Oracle? Efficient Strategies to Label Data](https://mlsanity.com/paper/2110.02341/) (2021)

## Abstract (arXiv)

Digital crowdsourcing (CS) is a modern approach to perform certain large projects using small contributions of a large crowd. In CS, a taskmaster typically breaks down the project into small batches of tasks and assigns them to so-called workers with imperfect skill levels. The crowdsourcer then collects and analyzes the results for inference and serving the purpose of the project. In this work, the CS problem, as a human-in-the-loop computation problem, is modeled and analyzed in an information theoretic rate-distortion framework. The purpose is to identify the ultimate fidelity that one can achieve by any form of query from the crowd and any decoding (inference) algorithm with a given budget. The results are established by a joint source channel (de)coding scheme, which represent the query scheme and inference, over parallel noisy channels, which model workers with imperfect skill levels. We also present and analyze a query scheme dubbed k-ary incidence coding and study optimized query pricing in this setting.