# Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions

arXiv: https://arxiv.org/abs/2412.06606 · HTML: https://mlsanity.com/paper/2412.06606/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings](https://mlsanity.com/paper/2202.06671/) (2022)
- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review](https://mlsanity.com/paper/1806.06237/) (2018)
- [Paper Matching with Local Fairness Constraints](https://mlsanity.com/paper/1905.11924/) (2019)
- [Simple and Effective Paraphrastic Similarity from Parallel Translations](https://mlsanity.com/paper/1909.13872/) (2019)
- [Matching Papers and Reviewers at Large Conferences](https://mlsanity.com/paper/2202.12273/) (2022)
- [Editable User Profiles for Controllable Text Recommendation](https://mlsanity.com/paper/2304.04250/) (2023)
- [Making Paper Reviewing Robust to Bid Manipulation Attacks](https://mlsanity.com/paper/2102.06020/) (2021)
- [I Will Have Order! Optimizing Orders for Fair Reviewer Assignment](https://mlsanity.com/paper/2108.02126/) (2021)
- [A Dataset on Malicious Paper Bidding in Peer Review](https://mlsanity.com/paper/2207.02303/) (2022)
- [No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial Learning](https://mlsanity.com/paper/2303.14443/) (2023)
- [On the Detection of Reviewer-Author Collusion Rings From Paper Bidding](https://mlsanity.com/paper/2402.07860/) (2024)
- [Combating Collusion Rings is Hard but Possible](https://mlsanity.com/paper/2112.08444/) (2021)
- [A One-Size-Fits-All Approach to Improving Randomness in Paper Assignment](https://mlsanity.com/paper/2310.05995/) (2023)
- [Tradeoffs in Preventing Manipulation in Paper Bidding for Reviewer Assignment](https://mlsanity.com/paper/2207.11315/) (2022)

## Built on by (most cited first, 7 of 7)

- [The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems](https://mlsanity.com/paper/2509.08713/) (2025)
- [Identity Theft in AI Conference Peer Review](https://mlsanity.com/paper/2508.04024/) (2025)
- [RATE: Reviewer Profiling and Annotation-free Training for Expertise Ranking in Peer Review Systems](https://mlsanity.com/paper/2601.19637/) (2026)
- [CABAL: Multi-Agent Simulacra for Tracing the Effects of Collusive Bidding in Peer Review](https://mlsanity.com/paper/2609.05227/) (2026)
- [Who is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition](https://mlsanity.com/paper/2510.12692/) (2025)
- [Smooth Partial Lotteries for Stable Randomized Selection](https://mlsanity.com/paper/2605.20069/) (2026)
- [MERIT: Matching Expertise via Rubric-Informed Training for Reviewer Assignment](https://mlsanity.com/paper/2605.27865/) (2026)

## Abstract (arXiv)

In the peer review process of top-tier machine learning (ML) and artificial intelligence (AI) conferences, reviewers are assigned to papers through automated methods. These assignment algorithms consider two main factors: (1) reviewers' expressed interests indicated by their bids for papers, and (2) reviewers' domain expertise inferred from the similarity between the text of their previously published papers and the submitted manuscripts. A significant challenge these conferences face is the existence of collusion rings, where groups of researchers manipulate the assignment process to review each other's papers, providing positive evaluations regardless of their actual quality. Most efforts to combat collusion rings have focused on preventing bid manipulation, under the assumption that the text similarity component is secure. In this paper, we demonstrate that even in the absence of bidding, colluding reviewers and authors can exploit the machine learning based text-matching component of reviewer assignment used at top ML/AI venues to get assigned their target paper. We also highlight specific vulnerabilities within this system and offer suggestions to enhance its robustness.