# Auctions and Peer Prediction for Academic Peer Review

arXiv: https://arxiv.org/abs/2109.00923 · HTML: https://mlsanity.com/paper/2109.00923/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 68 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 18 of 18)

- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [Design and Analysis of the NIPS 2016 Review Process](https://mlsanity.com/paper/1708.09794/) (2017)
- [Crowdsourced Judgement Elicitation with Endogenous Proficiency](https://mlsanity.com/paper/1303.0799/) (2013)
- [Optimum Statistical Estimation with Strategic Data Sources](https://mlsanity.com/paper/1408.2539/) (2014)
- [Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings](https://mlsanity.com/paper/1806.05085/) (2018)
- [On Strategyproof Conference Peer Review](https://mlsanity.com/paper/1806.06266/) (2018)
- [A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences](https://mlsanity.com/paper/2011.15050/) (2020)
- [Strategyproof Peer Selection using Randomization, Partitioning, and Apportionment](https://mlsanity.com/paper/1604.03632/) (2016)
- [Eliciting Expertise without Verification](https://mlsanity.com/paper/1802.08312/) (2018)
- [Loss Functions, Axioms, and Peer Review](https://mlsanity.com/paper/1808.09057/) (2018)
- [Avoiding a Tragedy of the Commons in the Peer Review Process](https://mlsanity.com/paper/1901.06246/) (2018)
- [A SUPER* Algorithm to Optimize Paper Bidding in Peer Review](https://mlsanity.com/paper/2007.07079/) (2020)
- [Learning to Incentivize: Eliciting Effort via Output Agreement](https://mlsanity.com/paper/1604.04928/) (2016)
- [Near-Optimal Reviewer Splitting in Two-Phase Paper Reviewing and Conference Experiment Design](https://mlsanity.com/paper/2108.06371/) (2021)
- [Incentive Design in Peer Review: Rating and Repeated Endogenous Matching](https://mlsanity.com/paper/1411.2139/) (2014)
- [Incentivizing Evaluation via Limited Access to Ground Truth: Peer-Prediction Makes Things Worse](https://mlsanity.com/paper/1606.07042/) (2016)
- [Inducing Honest Reporting Without Observing Outcomes: An Application to the Peer-Review Process](https://mlsanity.com/paper/1309.3197/) (2013)

## Built on by (most cited first, 2 of 2)

- [Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards](https://mlsanity.com/paper/2505.04966/) (2025)
- [Reimagining Peer Review Process Through Multi-Agent Mechanism Design](https://mlsanity.com/paper/2601.19778/) (2026)

## Abstract (arXiv)

Peer reviewed publications are considered the gold standard in certifying and disseminating ideas that a research community considers valuable. However, we identify two major drawbacks of the current system: (1) the overwhelming demand for reviewers due to a large volume of submissions, and (2) the lack of incentives for reviewers to participate and expend the necessary effort to provide high-quality reviews. In this work, we adopt a mechanism-design approach to propose improvements to the peer review process, tying together the paper submission and review processes and simultaneously incentivizing high-quality submissions and reviews. In the submission stage, authors participate in a VCG auction for review slots by submitting their papers along with a bid that represents their expected value for having their paper reviewed. For the reviewing stage, we propose a novel peer prediction mechanism (H-DIPP) building on recent work in the information elicitation literature, which incentivizes participating reviewers to provide honest and effortful reviews. The revenue raised in the submission stage auction is used to pay reviewers based on the quality of their reviews in the reviewing stage.