# Some Ethical Issues in the Review Process of Machine Learning Conferences

arXiv: https://arxiv.org/abs/2106.00810 · HTML: https://mlsanity.com/paper/2106.00810/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [Design and Analysis of the NIPS 2016 Review Process](https://mlsanity.com/paper/1708.09794/) (2017)
- [Does My Rebuttal Matter? Insights from a Major NLP Conference](https://mlsanity.com/paper/1903.11367/) (2019)
- [An Open Review of OpenReview: A Critical Analysis of the Machine Learning Conference Review Process](https://mlsanity.com/paper/2010.05137/) (2020)
- [A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences](https://mlsanity.com/paper/2011.15050/) (2020)
- [Avoiding a Tragedy of the Commons in the Peer Review Process](https://mlsanity.com/paper/1901.06246/) (2018)
- [De-anonymization of authors through arXiv submissions during double-blind review](https://mlsanity.com/paper/2007.00177/) (2020)

## Built on by (most cited first, 4 of 4)

- [Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards](https://mlsanity.com/paper/2505.04966/) (2025)
- [Automatic Analysis of Substantiation in Scientific Peer Reviews](https://mlsanity.com/paper/2311.11967/) (2023)
- [The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review](https://mlsanity.com/paper/2408.13430/) (2024)
- [Impact of large language models on peer review opinions from a fine-grained perspective: Evidence from top conference proceedings in AI](https://mlsanity.com/paper/2604.19578/) (2026)

## Abstract (arXiv)

Recent successes in the Machine Learning community have led to a steep increase in the number of papers submitted to conferences. This increase made more prominent some of the issues that affect the current review process used by these conferences. The review process has several issues that may undermine the nature of scientific research, which is of being fully objective, apolitical, unbiased and free of misconduct (such as plagiarism, cheating, improper influence, and other improprieties). In this work, we study the problem of reviewers' recruitment, infringements of the double-blind process, fraudulent behaviors, biases in numerical ratings, and the appendix phenomenon (i.e., the fact that it is becoming more common to publish results in the appendix section of a paper). For each of these problems, we provide a short description and possible solutions. The goal of this work is to raise awareness in the Machine Learning community regarding these issues.