# On the Detection of Reviewer-Author Collusion Rings From Paper Bidding

arXiv: https://arxiv.org/abs/2402.07860 · HTML: https://mlsanity.com/paper/2402.07860/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 155 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 30 of 43)

- [A Comprehensive Survey on Graph Anomaly Detection with Deep Learning](https://mlsanity.com/paper/2106.07178/) (2021)
- [Graph-based Anomaly Detection and Description: A Survey](https://mlsanity.com/paper/1404.4679/) (2014)
- [From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness](https://mlsanity.com/paper/2110.03753/) (2021)
- [Construction of the Literature Graph in Semantic Scholar](https://mlsanity.com/paper/1805.02262/) (2018)
- [On Using Classification Datasets to Evaluate Graph-Level Outlier Detection: Peculiar Observations and New Insights](https://mlsanity.com/paper/2012.12931/) (2020)
- [Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments](https://mlsanity.com/paper/2006.16437/) (2020)
- [Fast Memory-efficient Anomaly Detection in Streaming Heterogeneous Graphs](https://mlsanity.com/paper/1602.04844/) (2016)
- [Scalable Anomaly Ranking of Attributed Neighborhoods](https://mlsanity.com/paper/1601.06711/) (2016)
- [Design and Analysis of the NIPS 2016 Review Process](https://mlsanity.com/paper/1708.09794/) (2017)
- [Statistical Analysis of Nearest Neighbor Methods for Anomaly Detection](https://mlsanity.com/paper/1907.03813/) (2019)
- [Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solution](https://mlsanity.com/paper/2206.07647/) (2022)
- [Toward Unsupervised Outlier Model Selection](https://mlsanity.com/paper/2211.01834/) (2022)
- [Matching Papers and Reviewers at Large Conferences](https://mlsanity.com/paper/2202.12273/) (2022)
- [AutoAudit: Mining Accounting and Time-Evolving Graphs](https://mlsanity.com/paper/2011.00447/) (2020)
- [Catch Me if I Can: Detecting Strategic Behaviour in Peer Assessment](https://mlsanity.com/paper/2010.04041/) (2020)
- [Explaining Anomalies in Groups with Characterizing Subspace Rules](https://mlsanity.com/paper/1708.05929/) (2017)
- [A Quest for Structure: Jointly Learning the Graph Structure and Semi-Supervised Classification](https://mlsanity.com/paper/1909.12385/) (2019)
- [Making Paper Reviewing Robust to Bid Manipulation Attacks](https://mlsanity.com/paper/2102.06020/) (2021)
- [Data Augmentation is a Hyperparameter: Cherry-picked Self-Supervision for Unsupervised Anomaly Detection is Creating the Illusion of Success](https://mlsanity.com/paper/2208.07734/) (2022)
- [A Gold Standard Dataset for the Reviewer Assignment Problem](https://mlsanity.com/paper/2303.16750/) (2023)
- [Unified Discrete Diffusion for Categorical Data](https://mlsanity.com/paper/2402.03701/) (2024)
- [EdgeCentric: Anomaly Detection in Edge-Attributed Networks](https://mlsanity.com/paper/1510.05544/) (2015)
- [Automating Outlier Detection via Meta-Learning](https://mlsanity.com/paper/2009.10606/) (2020)
- [A Dataset on Malicious Paper Bidding in Peer Review](https://mlsanity.com/paper/2207.02303/) (2022)
- [Graph Anomaly Detection with Unsupervised GNNs](https://mlsanity.com/paper/2210.09535/) (2022)
- [No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial Learning](https://mlsanity.com/paper/2303.14443/) (2023)
- [Benefit-aware Early Prediction of Health Outcomes on Multivariate EEG Time Series](https://mlsanity.com/paper/2111.06032/) (2021)
- [Combating Collusion Rings is Hard but Possible](https://mlsanity.com/paper/2112.08444/) (2021)
- [DSV: An Alignment Validation Loss for Self-supervised Outlier Model Selection](https://mlsanity.com/paper/2307.06534/) (2023)
- [Fast Unsupervised Deep Outlier Model Selection with Hypernetworks](https://mlsanity.com/paper/2307.10529/) (2023)

## Built on by (most cited first, 7 of 7)

- [OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews](https://mlsanity.com/paper/2412.11948/) (2024)
- [Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards](https://mlsanity.com/paper/2505.04966/) (2025)
- [What Can Natural Language Processing Do for Peer Review?](https://mlsanity.com/paper/2405.06563/) (2024)
- [Insights from the ICLR Peer Review and Rebuttal Process](https://mlsanity.com/paper/2511.15462/) (2025)
- [Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions](https://mlsanity.com/paper/2412.06606/) (2024)
- [CABAL: Multi-Agent Simulacra for Tracing the Effects of Collusive Bidding in Peer Review](https://mlsanity.com/paper/2609.05227/) (2026)
- [Position: Want Better ML Reviews? Stop Asking Nicely and Start Incentivizing with a Credit System](https://mlsanity.com/paper/2608.14571/) (2026)

## Abstract (arXiv)

A major threat to the peer-review systems of computer science conferences is the existence of "collusion rings" between reviewers. In such collusion rings, reviewers who have also submitted their own papers to the conference work together to manipulate the conference's paper assignment, with the aim of being assigned to review each other's papers. The most straightforward way that colluding reviewers can manipulate the paper assignment is by indicating their interest in each other's papers through strategic paper bidding. One potential approach to solve this important problem would be to detect the colluding reviewers from their manipulated bids, after which the conference can take appropriate action. While prior work has developed effective techniques to detect other kinds of fraud, no research has yet established that detecting collusion rings is even possible. In this work, we tackle the question of whether it is feasible to detect collusion rings from the paper bidding. To answer this question, we conduct empirical analysis of two realistic conference bidding datasets, including evaluations of existing algorithms for fraud detection in other applications. We find that collusion rings can achieve considerable success at manipulating the paper assignment while remaining hidden from detection: for example, in one dataset, undetected colluders are able to achieve assignment to up to 30% of the papers authored by other colluders. In addition, when 10 colluders bid on all of each other's papers, no detection algorithm outputs a group of reviewers with more than 31% overlap with the true colluders. These results suggest that collusion cannot be effectively detected from the bidding using popular existing tools, demonstrating the need to develop more complex detection algorithms as well as those that leverage additional metadata (e.g., reviewer-paper text-similarity scores).