# No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial Learning

arXiv: https://arxiv.org/abs/2303.14443 · HTML: https://mlsanity.com/paper/2303.14443/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 65 references that are in the map.
- First posted on arXiv 25 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Crafting Adversarial Input Sequences for Recurrent Neural Networks](https://mlsanity.com/paper/1604.08275/) (2016)
- [Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems](https://mlsanity.com/paper/1903.11508/) (2019)
- [Bad Characters: Imperceptible NLP Attacks](https://mlsanity.com/paper/2106.09898/) (2021)
- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [Misleading Authorship Attribution of Source Code using Adversarial Learning](https://mlsanity.com/paper/1905.12386/) (2019)
- [PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review](https://mlsanity.com/paper/1806.06237/) (2018)
- [Matching Papers and Reviewers at Large Conferences](https://mlsanity.com/paper/2202.12273/) (2022)
- [Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals](https://mlsanity.com/paper/2107.06751/) (2021)
- [What is Wrong with Topic Modeling? (and How to Fix it Using Search-based Software Engineering)](https://mlsanity.com/paper/1608.08176/) (2016)
- [Making Paper Reviewing Robust to Bid Manipulation Attacks](https://mlsanity.com/paper/2102.06020/) (2021)
- [Dompteur: Taming Audio Adversarial Examples](https://mlsanity.com/paper/2102.05431/) (2021)
- [Measuring LDA Topic Stability from Clusters of Replicated Runs](https://mlsanity.com/paper/1808.08098/) (2018)
- [EvaLDA: Efficient Evasion Attacks Towards Latent Dirichlet Allocation](https://mlsanity.com/paper/2012.04864/) (2020)

## Built on by (most cited first, 7 of 7)

- [The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems](https://mlsanity.com/paper/2509.08713/) (2025)
- [Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experiment](https://mlsanity.com/paper/2411.03417/) (2024)
- [Stop Automating Peer Review Without Rigorous Evaluation](https://mlsanity.com/paper/2605.03202/) (2026)
- [On the Detection of Reviewer-Author Collusion Rings From Paper Bidding](https://mlsanity.com/paper/2402.07860/) (2024)
- [Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions](https://mlsanity.com/paper/2412.06606/) (2024)
- [AttackGNN: Red-Teaming GNNs in Hardware Security Using Reinforcement Learning](https://mlsanity.com/paper/2402.13946/) (2024)
- [Who is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition](https://mlsanity.com/paper/2510.12692/) (2025)

## Abstract (arXiv)

The number of papers submitted to academic conferences is steadily rising in many scientific disciplines. To handle this growth, systems for automatic paper-reviewer assignments are increasingly used during the reviewing process. These systems use statistical topic models to characterize the content of submissions and automate the assignment to reviewers. In this paper, we show that this automation can be manipulated using adversarial learning. We propose an attack that adapts a given paper so that it misleads the assignment and selects its own reviewers. Our attack is based on a novel optimization strategy that alternates between the feature space and problem space to realize unobtrusive changes to the paper. To evaluate the feasibility of our attack, we simulate the paper-reviewer assignment of an actual security conference (IEEE S&P) with 165 reviewers on the program committee. Our results show that we can successfully select and remove reviewers without access to the assignment system. Moreover, we demonstrate that the manipulated papers remain plausible and are often indistinguishable from benign submissions.