# Catch Me if I Can: Detecting Strategic Behaviour in Peer Assessment

arXiv: https://arxiv.org/abs/2010.04041 · HTML: https://mlsanity.com/paper/2010.04041/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 14 of 14)

- [ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing](https://mlsanity.com/paper/2306.00622/) (2023)
- [What Can Natural Language Processing Do for Peer Review?](https://mlsanity.com/paper/2405.06563/) (2024)
- [Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyond](https://mlsanity.com/paper/2201.11443/) (2022)
- [Insights from the ICLR Peer Review and Rebuttal Process](https://mlsanity.com/paper/2511.15462/) (2025)
- [On the Detection of Reviewer-Author Collusion Rings From Paper Bidding](https://mlsanity.com/paper/2402.07860/) (2024)
- [Near-Optimal Reviewer Splitting in Two-Phase Paper Reviewing and Conference Experiment Design](https://mlsanity.com/paper/2108.06371/) (2021)
- [Strategyproofing Peer Assessment via Partitioning: The Price in Terms of Evaluators' Expertise](https://mlsanity.com/paper/2201.10631/) (2022)
- [Least Square Calibration for Peer Review](https://mlsanity.com/paper/2110.12607/) (2021)
- [Peer Selection with Noisy Assessments](https://mlsanity.com/paper/2107.10121/) (2021)
- [Enhancing Peer Review in Astronomy: A Machine Learning and Optimization Approach to Reviewer Assignments for ALMA](https://mlsanity.com/paper/2410.10009/) (2024)
- [Manipulation and Peer Mechanisms: A Survey](https://mlsanity.com/paper/2210.01984/) (2022)
- [Improving Peer Assessment with Graph Convolutional Networks](https://mlsanity.com/paper/2111.04466/) (2021)
- [No Agreement Without Loss: Learning and Social Choice in Peer Review](https://mlsanity.com/paper/2211.02144/) (2022)
- [Articulate Intuition or Genuine Analysis? Benchmarking Epistemic Reliability in LLM-as-a-Judge Peer Reviews](https://mlsanity.com/paper/2607.10511/) (2026)

## Abstract (arXiv)

We consider the issue of strategic behaviour in various peer-assessment tasks, including peer grading of exams or homeworks and peer review in hiring or promotions. When a peer-assessment task is competitive (e.g., when students are graded on a curve), agents may be incentivized to misreport evaluations in order to improve their own final standing. Our focus is on designing methods for detection of such manipulations. Specifically, we consider a setting in which agents evaluate a subset of their peers and output rankings that are later aggregated to form a final ordering. In this paper, we investigate a statistical framework for this problem and design a principled test for detecting strategic behaviour. We prove that our test has strong false alarm guarantees and evaluate its detection ability in practical settings. For this, we design and execute an experiment that elicits strategic behaviour from subjects and release a dataset of patterns of strategic behaviour that may be of independent interest. We then use the collected data to conduct a series of real and semi-synthetic evaluations that demonstrate a strong detection power of our test.