# PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review

arXiv: https://arxiv.org/abs/1806.06237 · HTML: https://mlsanity.com/paper/1806.06237/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2018.

## Built on by (most cited first, 30 of 30)

- [ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing](https://mlsanity.com/paper/2306.00622/) (2023)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [What Can Natural Language Processing Do for Peer Review?](https://mlsanity.com/paper/2405.06563/) (2024)
- [Matching Papers and Reviewers at Large Conferences](https://mlsanity.com/paper/2202.12273/) (2022)
- [Editable User Profiles for Controllable Text Recommendation](https://mlsanity.com/paper/2304.04250/) (2023)
- [Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experiment](https://mlsanity.com/paper/2411.03417/) (2024)
- [How Far Are AI Scientists from Changing the World?](https://mlsanity.com/paper/2507.23276/) (2025)
- [Making Paper Reviewing Robust to Bid Manipulation Attacks](https://mlsanity.com/paper/2102.06020/) (2021)
- [A Gold Standard Dataset for the Reviewer Assignment Problem](https://mlsanity.com/paper/2303.16750/) (2023)
- [Chain-of-Factors Paper-Reviewer Matching](https://mlsanity.com/paper/2310.14483/) (2023)
- [No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial Learning](https://mlsanity.com/paper/2303.14443/) (2023)
- [Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions](https://mlsanity.com/paper/2412.06606/) (2024)
- [Near-Optimal Reviewer Splitting in Two-Phase Paper Reviewing and Conference Experiment Design](https://mlsanity.com/paper/2108.06371/) (2021)
- [Combating Collusion Rings is Hard but Possible](https://mlsanity.com/paper/2112.08444/) (2021)
- [Strategyproofing Peer Assessment via Partitioning: The Price in Terms of Evaluators' Expertise](https://mlsanity.com/paper/2201.10631/) (2022)
- [Group Fairness in Peer Review](https://mlsanity.com/paper/2410.03474/) (2024)
- [What Factors Should Paper-Reviewer Assignments Rely On? Community Perspectives on Issues and Ideals in Conference Peer-Review](https://mlsanity.com/paper/2205.01005/) (2022)
- [Fairness through Social Welfare Optimization](https://mlsanity.com/paper/2102.00311/) (2021)
- [Reviewer Scores Are Not Comparable Across Research Areas in ML Peer Review](https://mlsanity.com/paper/2607.27209/) (2026)
- [Matchings, Predictions and Counterfactual Harm in Refugee Resettlement Processes](https://mlsanity.com/paper/2407.13052/) (2024)
- [Harm Ratio: A Novel and Versatile Fairness Criterion](https://mlsanity.com/paper/2410.02977/) (2024)
- [Fair and Welfare-Efficient Constrained Multi-matchings under Uncertainty](https://mlsanity.com/paper/2411.02654/) (2024)
- [What happens when reviewers receive AI feedback in their reviews?](https://mlsanity.com/paper/2602.13817/) (2026)
- [CABAL: Multi-Agent Simulacra for Tracing the Effects of Collusive Bidding in Peer Review](https://mlsanity.com/paper/2609.05227/) (2026)
- [Multiple Fairness and Cardinality constraints for Students-Topics Grouping Problem](https://mlsanity.com/paper/2206.09895/) (2022)
- [Recovering Top-Two Answers and Confusion Probability in Multi-Choice Crowdsourcing](https://mlsanity.com/paper/2301.00006/) (2022)
- [Who is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition](https://mlsanity.com/paper/2510.12692/) (2025)
- [Which Coauthor Should I Nominate in My 99 ICLR Submissions? A Mathematical Analysis of the ICLR 2026 Reciprocal Reviewer Nomination Policy](https://mlsanity.com/paper/2512.17950/) (2025)
- [Allocate Marginal Reviews to Borderline Papers Using LLM Comparative Ranking](https://mlsanity.com/paper/2602.06078/) (2026)
- [The IJCNN 2025 Review Process](https://mlsanity.com/paper/2603.19244/) (2026)

## Abstract (arXiv)

We consider the problem of automated assignment of papers to reviewers in conference peer review, with a focus on fairness and statistical accuracy. Our fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the commonly used objective of maximizing the total quality over all papers. We design an assignment algorithm based on an incremental max-flow procedure that we prove is near-optimally fair. Our statistical accuracy objective is to ensure correct recovery of the papers that should be accepted. We provide a sharp minimax analysis of the accuracy of the peer-review process for a popular objective-score model as well as for a novel subjective-score model that we propose in the paper. Our analysis proves that our proposed assignment algorithm also leads to a near-optimal statistical accuracy. Finally, we design a novel experiment that allows for an objective comparison of various assignment algorithms, and overcomes the inherent difficulty posed by the absence of a ground truth in experiments on peer-review. The results of this experiment as well as of other experiments on synthetic and real data corroborate the theoretical guarantees of our algorithm.