# Eliciting Honest Information From Authors Using Sequential Review

arXiv: https://arxiv.org/abs/2311.14619 · HTML: https://mlsanity.com/paper/2311.14619/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 24 Nov 2023.

## Builds on (most cited first, 13 of 13)

- [Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment](https://mlsanity.com/paper/2109.09774/) (2021)
- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings](https://mlsanity.com/paper/1806.05085/) (2018)
- [You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism](https://mlsanity.com/paper/2110.14802/) (2021)
- [On Strategyproof Conference Peer Review](https://mlsanity.com/paper/1806.06266/) (2018)
- [Strategyproof Peer Selection using Randomization, Partitioning, and Apportionment](https://mlsanity.com/paper/1604.03632/) (2016)
- [Avoiding a Tragedy of the Commons in the Peer Review Process](https://mlsanity.com/paper/1901.06246/) (2018)
- [A System-Level Analysis of Conference Peer Review](https://mlsanity.com/paper/2303.09020/) (2023)
- [Strategyproofing Peer Assessment via Partitioning: The Price in Terms of Evaluators' Expertise](https://mlsanity.com/paper/2201.10631/) (2022)
- [Isotonic Mechanism for Exponential Family Estimation in Machine Learning Peer Review](https://mlsanity.com/paper/2304.11160/) (2023)
- [An Isotonic Mechanism for Overlapping Ownership](https://mlsanity.com/paper/2306.11154/) (2023)
- [Incentives, Quality, and Risks: A Look Into the NSF Proposal Review Pilot](https://mlsanity.com/paper/1307.6528/) (2013)
- [Perceptions and Truth: A Mechanism Design Approach to Crowd-Sourcing Reputation](https://mlsanity.com/paper/1306.0173/) (2013)

## Abstract (arXiv)

In the setting of conference peer review, the conference aims to accept high-quality papers and reject low-quality papers based on noisy review scores. A recent work proposes the isotonic mechanism, which can elicit the ranking of paper qualities from an author with multiple submissions to help improve the conference's decisions. However, the isotonic mechanism relies on the assumption that the author's utility is both an increasing and a convex function with respect to the review score, which is often violated in peer review settings (e.g.~when authors aim to maximize the number of accepted papers). In this paper, we propose a sequential review mechanism that can truthfully elicit the ranking information from authors while only assuming the agent's utility is increasing with respect to the true quality of her accepted papers. The key idea is to review the papers of an author in a sequence based on the provided ranking and conditioning the review of the next paper on the review scores of the previous papers. Advantages of the sequential review mechanism include 1) eliciting truthful ranking information in a more realistic setting than prior work; 2) improving the quality of accepted papers, reducing the reviewing workload and increasing the average quality of papers being reviewed; 3) incentivizing authors to write fewer papers of higher quality.