# You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism

arXiv: https://arxiv.org/abs/2110.14802 · HTML: https://mlsanity.com/paper/2110.14802/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 27 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [SLOPE - Adaptive variable selection via convex optimization](https://mlsanity.com/paper/1407.3824/) (2014)
- [What Can We Do to Improve Peer Review in NLP?](https://mlsanity.com/paper/2010.03863/) (2020)
- [A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences](https://mlsanity.com/paper/2011.15050/) (2020)
- [On Testing for Biases in Peer Review](https://mlsanity.com/paper/1912.13188/) (2019)
- [Strategyproof Peer Selection using Randomization, Partitioning, and Apportionment](https://mlsanity.com/paper/1604.03632/) (2016)
- [Loss Functions, Axioms, and Peer Review](https://mlsanity.com/paper/1808.09057/) (2018)
- [Avoiding a Tragedy of the Commons in the Peer Review Process](https://mlsanity.com/paper/1901.06246/) (2018)
- [Debiasing Evaluations That are Biased by Evaluations](https://mlsanity.com/paper/2012.00714/) (2020)
- [PeerNomination: Relaxing Exactness for Increased Accuracy in Peer Selection](https://mlsanity.com/paper/2004.14939/) (2020)
- [Arbitrariness of peer review: A Bayesian analysis of the NIPS experiment](https://mlsanity.com/paper/1507.06411/) (2015)
- [Incentive-Compatible Selection Mechanisms for Forests](https://mlsanity.com/paper/2006.00600/) (2020)

## Built on by (most cited first, 19 of 19)

- [GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study](https://mlsanity.com/paper/2307.05492/) (2023)
- [Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards](https://mlsanity.com/paper/2505.04966/) (2025)
- [From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Review](https://mlsanity.com/paper/2506.11343/) (2025)
- [Experimental Standards for Deep Learning in Natural Language Processing Research](https://mlsanity.com/paper/2204.06251/) (2022)
- [The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review](https://mlsanity.com/paper/2408.13430/) (2024)
- [How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?](https://mlsanity.com/paper/2211.12966/) (2022)
- [Isotonic Mechanism for Exponential Family Estimation in Machine Learning Peer Review](https://mlsanity.com/paper/2304.11160/) (2023)
- [How to Find Fantastic AI Papers: Self-Rankings as a Powerful Predictor of Scientific Impact Beyond Peer Review](https://mlsanity.com/paper/2510.02143/) (2025)
- [The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning](https://mlsanity.com/paper/2603.15914/) (2026)
- [Ranking Differential Privacy](https://mlsanity.com/paper/2301.00841/) (2023)
- [Recommending Best Paper Awards for ML/AI Conferences via the Isotonic Mechanism](https://mlsanity.com/paper/2601.15249/) (2026)
- [Strategic Candidacy in Generative AI Arenas](https://mlsanity.com/paper/2603.26891/) (2026)
- [Eliciting Honest Information From Authors Using Sequential Review](https://mlsanity.com/paper/2311.14619/) (2023)
- [From Authors to Reviewers: Leveraging Rankings to Improve Peer Review](https://mlsanity.com/paper/2510.21726/) (2025)
- [Reimagining Peer Review Process Through Multi-Agent Mechanism Design](https://mlsanity.com/paper/2601.19778/) (2026)
- [Allocate Marginal Reviews to Borderline Papers Using LLM Comparative Ranking](https://mlsanity.com/paper/2602.06078/) (2026)
- [CAIRO: Decoupling Order from Scale in Regression](https://mlsanity.com/paper/2602.14440/) (2026)
- [Aggregate-then-Calibrate for Human-centered Assessment with Theoretical Guarantees](https://mlsanity.com/paper/2608.02455/) (2026)
- [Position: Want Better ML Reviews? Stop Asking Nicely and Start Incentivizing with a Credit System](https://mlsanity.com/paper/2608.14571/) (2026)

## Abstract (arXiv)

I consider a setting where reviewers offer very noisy scores for several items for the selection of high-quality ones (e.g., peer review of large conference proceedings), whereas the owner of these items knows the true underlying scores but prefers not to provide this information. To address this withholding of information, in this paper, I introduce the Isotonic Mechanism, a simple and efficient approach to improving imprecise raw scores by leveraging certain information that the owner is incentivized to provide. This mechanism takes the ranking of the items from best to worst provided by the owner as input, in addition to the raw scores provided by the reviewers. It reports the adjusted scores for the items by solving a convex optimization problem. Under certain conditions, I show that the owner's optimal strategy is to honestly report the true ranking of the items to her best knowledge in order to maximize the expected utility. Moreover, I prove that the adjusted scores provided by this owner-assisted mechanism are significantly more accurate than the raw scores provided by the reviewers. This paper concludes with several extensions of the Isotonic Mechanism and some refinements of the mechanism for practical consideration.