# Learning and Strongly Truthful Multi-Task Peer Prediction: A Variational Approach

arXiv: https://arxiv.org/abs/2009.14730 · HTML: https://mlsanity.com/paper/2009.14730/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Multi-agent Performative Prediction: From Global Stability and Optimality to Chaos](https://mlsanity.com/paper/2201.10483/) (2022)
- [Eliciting Informative Text Evaluations with Large Language Models](https://mlsanity.com/paper/2405.15077/) (2024)
- [Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms](https://mlsanity.com/paper/2402.13567/) (2024)
- [Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth](https://mlsanity.com/paper/2506.06991/) (2025)
- [Stochastically Dominant Peer Prediction](https://mlsanity.com/paper/2506.02259/) (2025)
- [Incentive-Aligned Multi-Source LLM Summaries](https://mlsanity.com/paper/2509.25184/) (2025)
- [Let's Measure Information Step-by-Step: AI-Based Evaluation Beyond Vibes](https://mlsanity.com/paper/2508.05469/) (2025)
- [PMIScore: An Unsupervised Approach to Quantify Dialogue Engagement](https://mlsanity.com/paper/2603.13796/) (2026)

## Abstract (arXiv)

Peer prediction mechanisms incentivize agents to truthfully report their signals even in the absence of verification by comparing agents' reports with those of their peers. In the detail-free multi-task setting, agents respond to multiple independent and identically distributed tasks, and the mechanism does not know the prior distribution of agents' signals. The goal is to provide an ε-strongly truthful mechanism where truth-telling rewards agents "strictly" more than any other strategy profile (with ε additive error), and to do so while requiring as few tasks as possible. We design a family of mechanisms with a scoring function that maps a pair of reports to a score. The mechanism is strongly truthful if the scoring function is "prior ideal," and ε-strongly truthful as long as the scoring function is sufficiently close to the ideal one. This reduces the above mechanism design problem to a learning problem -- specifically learning an ideal scoring function. We leverage this reduction to obtain the following three results. 1) We show how to derive good bounds on the number of tasks required for different types of priors. Our reduction applies to myriad continuous signal space settings. This is the first peer-prediction mechanism on continuous signals designed for the multi-task setting. 2) We show how to turn a soft-predictor of an agent's signals (given the other agents' signals) into a mechanism. This allows the practical use of machine learning algorithms that give good results even when many agents provide noisy information. 3) For finite signal spaces, we obtain ε-strongly truthful mechanisms on any stochastically relevant prior, which is the maximal possible prior. In contrast, prior work only achieves a weaker notion of truthfulness (informed truthfulness) or requires stronger assumptions on the prior.