# Isotonic Mechanism for Exponential Family Estimation in Machine Learning Peer Review

arXiv: https://arxiv.org/abs/2304.11160 · HTML: https://mlsanity.com/paper/2304.11160/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 21 Apr 2023.

## Builds on (most cited first, 7 of 7)

- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [Paper Matching with Local Fairness Constraints](https://mlsanity.com/paper/1905.11924/) (2019)
- [Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings](https://mlsanity.com/paper/1806.05085/) (2018)
- [You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism](https://mlsanity.com/paper/2110.14802/) (2021)
- [Matching Papers and Reviewers at Large Conferences](https://mlsanity.com/paper/2202.12273/) (2022)
- [A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences](https://mlsanity.com/paper/2011.15050/) (2020)
- [Calibration with confidence: A principled method for panel assessment](https://mlsanity.com/paper/1504.00340/) (2015)

## Built on by (most cited first, 6 of 6)

- [From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Review](https://mlsanity.com/paper/2506.11343/) (2025)
- [The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review](https://mlsanity.com/paper/2408.13430/) (2024)
- [How to Find Fantastic AI Papers: Self-Rankings as a Powerful Predictor of Scientific Impact Beyond Peer Review](https://mlsanity.com/paper/2510.02143/) (2025)
- [Recommending Best Paper Awards for ML/AI Conferences via the Isotonic Mechanism](https://mlsanity.com/paper/2601.15249/) (2026)
- [Eliciting Honest Information From Authors Using Sequential Review](https://mlsanity.com/paper/2311.14619/) (2023)
- [CAIRO: Decoupling Order from Scale in Regression](https://mlsanity.com/paper/2602.14440/) (2026)

## Abstract (arXiv)

In 2023, the International Conference on Machine Learning (ICML) required authors with multiple submissions to rank their submissions based on perceived quality. In this paper, we aim to employ these author-specified rankings to enhance peer review in machine learning and artificial intelligence conferences by extending the Isotonic Mechanism to exponential family distributions. This mechanism generates adjusted scores that closely align with the original scores while adhering to author-specified rankings. Despite its applicability to a broad spectrum of exponential family distributions, implementing this mechanism does not require knowledge of the specific distribution form. We demonstrate that an author is incentivized to provide accurate rankings when her utility takes the form of a convex additive function of the adjusted review scores. For a certain subclass of exponential family distributions, we prove that the author reports truthfully only if the question involves only pairwise comparisons between her submissions, thus indicating the optimality of ranking in truthful information elicitation. Moreover, we show that the adjusted scores improve dramatically the estimation accuracy compared to the original scores and achieve nearly minimax optimality when the ground-truth scores have bounded total variation. We conclude with a numerical analysis of the ICML 2023 ranking data, showing substantial estimation gains in approximating a proxy ground-truth quality of the papers using the Isotonic Mechanism.