# Informed Truthfulness in Multi-Task Peer Prediction

arXiv: https://arxiv.org/abs/1603.03151 · HTML: https://mlsanity.com/paper/1603.03151/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2016.

## Built on by (most cited first, 25 of 25)

- [Data Pricing in Machine Learning Pipelines](https://mlsanity.com/paper/2108.07915/) (2021)
- [Eliciting Informative Text Evaluations with Large Language Models](https://mlsanity.com/paper/2405.15077/) (2024)
- [Data-Free Evaluation of User Contributions in Federated Learning](https://mlsanity.com/paper/2108.10623/) (2021)
- [Learning from Pixel-Level Noisy Label : A New Perspective for Light Field Saliency Detection](https://mlsanity.com/paper/2204.13456/) (2022)
- [SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing](https://mlsanity.com/paper/2509.11265/) (2025)
- [Surprisingly Popular Voting Recovers Rankings, Surprisingly!](https://mlsanity.com/paper/2105.09386/) (2021)
- [Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms](https://mlsanity.com/paper/2402.13567/) (2024)
- [Reward-Based 1-bit Compressed Federated Distillation on Blockchain](https://mlsanity.com/paper/2106.14265/) (2021)
- [Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth](https://mlsanity.com/paper/2506.06991/) (2025)
- [Better Peer Grading through Bayesian Inference](https://mlsanity.com/paper/2209.01242/) (2022)
- [Truthful Generalized Linear Models](https://mlsanity.com/paper/2209.07815/) (2022)
- [On The Truthfulness of 'Surprisingly Likely' Responses of Large Language Models](https://mlsanity.com/paper/2311.07692/) (2023)
- [Navigating Towards Fairness with Data Selection](https://mlsanity.com/paper/2412.11072/) (2024)
- [Stochastically Dominant Peer Prediction](https://mlsanity.com/paper/2506.02259/) (2025)
- [Incentive-Aligned Multi-Source LLM Summaries](https://mlsanity.com/paper/2509.25184/) (2025)
- [Incentivizing an Unknown Crowd](https://mlsanity.com/paper/2109.04226/) (2021)
- [Eliciting Truthful Reports with Partial Signals in Repeated Games](https://mlsanity.com/paper/2109.04343/) (2021)
- [Information Elicitation Meets Clustering](https://mlsanity.com/paper/2110.00952/) (2021)
- [Improving Peer Assessment with Graph Convolutional Networks](https://mlsanity.com/paper/2111.04466/) (2021)
- [Bayes classifier cannot be learned from noisy responses with unknown noise rates](https://mlsanity.com/paper/2304.06574/) (2023)
- [Incentive-Compatible Recovery from Manipulated Signals, with Applications to Decentralized Physical Infrastructure](https://mlsanity.com/paper/2503.07558/) (2025)
- [Incentivizing High-quality Participation From Federated Learning Agents](https://mlsanity.com/paper/2506.16731/) (2025)
- [Multi-agent Adaptive Mechanism Design](https://mlsanity.com/paper/2512.21794/) (2025)
- [Knowledge-Free Correlated Agreement for Incentivizing Federated Learning](https://mlsanity.com/paper/2605.04747/) (2026)
- [Mitigating LLM sycophancy with RL-based fine-tuning: Bayesian Truth Serum approach](https://mlsanity.com/paper/2608.25267/) (2026)

## Abstract (arXiv)

The problem of peer prediction is to elicit information from agents in settings without any objective ground truth against which to score reports. Peer prediction mechanisms seek to exploit correlations between signals to align incentives with truthful reports. A long-standing concern has been the possibility of uninformative equilibria. For binary signals, a multi-task mechanism [Dasgupta-Ghosh '13] achieves strong truthfulness, so that the truthful equilibrium strictly maximizes payoff. We characterize conditions on the signal distribution for which this mechanism remains strongly-truthful with non-binary signals, also providing a greatly simplified proof. We introduce the Correlated Agreement (CA) mechanism, which handles multiple signals and provides informed truthfulness: no strategy profile provides more payoff in equilibrium than truthful reporting, and the truthful equilibrium is strictly better than any uninformed strategy (where an agent avoids the effort of obtaining a signal). The CA mechanism is maximally strongly truthful, in that no mechanism in a broad class of mechanisms is strongly truthful on a larger family of signal distributions. We also give a detail-free version of the mechanism that removes any knowledge requirements on the part of the designer, using reports on many tasks to learn statistics while retaining epsilon-informed truthfulness.