# Recommending Best Paper Awards for ML/AI Conferences via the Isotonic Mechanism

arXiv: https://arxiv.org/abs/2601.15249 · HTML: https://mlsanity.com/paper/2601.15249/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Builds on (most cited first, 15 of 15)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Can We Automate Scientific Reviewing?](https://mlsanity.com/paper/2102.00176/) (2021)
- [Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment](https://mlsanity.com/paper/2109.09774/) (2021)
- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment](https://mlsanity.com/paper/2306.03262/) (2023)
- [Paper Matching with Local Fairness Constraints](https://mlsanity.com/paper/1905.11924/) (2019)
- [Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings](https://mlsanity.com/paper/1806.05085/) (2018)
- [No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"](https://mlsanity.com/paper/2209.14987/) (2022)
- [You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism](https://mlsanity.com/paper/2110.14802/) (2021)
- [Matching Papers and Reviewers at Large Conferences](https://mlsanity.com/paper/2202.12273/) (2022)
- [The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review](https://mlsanity.com/paper/2408.13430/) (2024)
- [A Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions](https://mlsanity.com/paper/2011.15083/) (2020)
- [Isotonic Mechanism for Exponential Family Estimation in Machine Learning Peer Review](https://mlsanity.com/paper/2304.11160/) (2023)
- [An Isotonic Mechanism for Overlapping Ownership](https://mlsanity.com/paper/2306.11154/) (2023)
- [How to Find Fantastic AI Papers: Self-Rankings as a Powerful Predictor of Scientific Impact Beyond Peer Review](https://mlsanity.com/paper/2510.02143/) (2025)

## Built on by (most cited first, 1 of 1)

- [Rejoinder: The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review](https://mlsanity.com/paper/2605.25172/) (2026)

## Abstract (arXiv)

Machine learning and artificial intelligence conferences such as NeurIPS and ICML now regularly receive tens of thousands of submissions, posing significant challenges to maintaining the quality and consistency of the peer review process. This challenge is particularly acute for best paper awards, which are an important part of the peer review process, yet whose selection has increasingly become a subject of debate in recent years. In this paper, we introduce an author-assisted mechanism to facilitate the selection of best paper awards. Our method employs the Isotonic Mechanism for eliciting authors' assessments of their own submissions in the form of a ranking, which is subsequently utilized to adjust the raw review scores for optimal estimation of the submissions' ground-truth quality. We demonstrate that authors are incentivized to report truthfully when their utility is a convex additive function of the adjusted scores, and we validate this convexity assumption for best paper awards using publicly accessible review data of ICLR from 2019 to 2023 and NeurIPS from 2021 to 2023. Crucially, in the special case where an author has a single quota -- that is, may nominate only one paper -- we prove that truthfulness holds even when the utility function is merely nondecreasing and additive. This finding represents a substantial relaxation of the assumptions required in prior work. For practical implementation, we extend our mechanism to accommodate the common scenario of overlapping authorship. Finally, simulation results demonstrate that our mechanism significantly improves the quality of papers selected for awards.