Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks
Not on the map. Why?
Abstract (arXiv)
In the setting where participants are asked multiple similar possibly subjective multi-choice questions (e.g. Do you like Panda Express? Y/N; do you like Chick-fil-A? Y/N), a series of peer prediction mechanisms are designed to incentivize honest reports and some of them achieve dominantly truthfulness: truth-telling is a dominant strategy and strictly dominate other "non-permutation strategy" with some mild conditions. However, a major issue hinders the practical usage of those mechanisms: they require the participants to perform an infinite number of tasks. When the participants perform a finite number of tasks, these mechanisms only achieve approximated dominant truthfulness. The existence of a dominantly truthful multi-task peer prediction mechanism that only requires a finite number of tasks remains to be an open question that may have a negative result, even with full prior knowledge. This paper answers this open question by proposing a new mechanism, Determinant based Mutual Information Mechanism (DMI-Mechanism), that is dominantly truthful when the number of tasks is at least 2C and the number of participants is at least 2. C is the number of choices for each question (C=2 for binary-choice questions). In addition to incentivizing honest reports, DMI-Mechanism can also be transferred into an information evaluation rule that identifies high-quality information without verification when there are at least 3 participants. To the best of our knowledge, DMI-Mechanism is the first dominantly truthful mechanism that works for a finite number of tasks, not to say a small constant number of tasks.
Key facts. Cited by 12 papers in the map (data through 13 Oct 2026). First posted on arXiv 1 Nov 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 1; 2023: 0; 2024: 2; 2025: 4; 2026: 2.
12 papers.
- 2021A Bayesian Framework for Information-Theoretic Probingcited by 22
- 2024Eliciting Informative Text Evaluations with Large Language Modelscited by 15
- 2025Incentivizing Truthful Language Models via Peer Elicitation Gamescited by 9
- 2022Robust Local Preserving and Global Aligning Network for Adversarial Domain Adaptationcited by 8
- 2021Surprisingly Popular Voting Recovers Rankings, Surprisingly!cited by 4
- 2024Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanismscited by 4
- 2025Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truthcited by 2
- 2025Data Reliability Scoringcited by 2
- 2026Truthfulness Despite Weak Supervision: Evaluating and Training LLMs Using Peer Predictioncited by 2
- 2025Stochastically Dominant Peer Predictioncited by 1
- 2021Information Elicitation Meets Clusteringcited by 0
- 2026Scoring Rules! Statistical and Strategic Alignment for Text Evaluation Metricscited by 0
- 2026Scoring Rules! Statistical and Strategic Alignment for Text Evaluation Metrics2 Aug 2026
- 2026Truthfulness Despite Weak Supervision: Evaluating and Training LLMs Using Peer Prediction28 Jan 2026
- 2025Data Reliability Scoring20 Oct 2025
- 2025Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth8 Jun 2025
- 2025Stochastically Dominant Peer Prediction2 Jun 2025
- 2025Incentivizing Truthful Language Models via Peer Elicitation Games19 May 2025
- 2024Eliciting Informative Text Evaluations with Large Language Models23 May 2024
- 2024Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms21 Feb 2024
- 2022Robust Local Preserving and Global Aligning Network for Adversarial Domain Adaptation8 Mar 2022
- 2021Information Elicitation Meets Clustering3 Oct 2021
- 2021A Bayesian Framework for Information-Theoretic Probing8 Sep 2021
- 2021Surprisingly Popular Voting Recovers Rankings, Surprisingly!19 May 2021