# PeerPrism: Peer Evaluation Expertise vs Review-writing AI

arXiv: https://arxiv.org/abs/2604.14513 · HTML: https://mlsanity.com/paper/2604.14513/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Builds on (most cited first, 21 of 21)

- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [Testing of Detection Tools for AI-Generated Text](https://mlsanity.com/paper/2306.15666/) (2023)
- [Can We Automate Scientific Reviewing?](https://mlsanity.com/paper/2102.00176/) (2021)
- [Large Language Models are Diverse Role-Players for Summarization Evaluation](https://mlsanity.com/paper/2303.15078/) (2023)
- [Do LLMs write like humans? Variation in grammatical and rhetorical styles](https://mlsanity.com/paper/2410.16107/) (2024)
- [Training-free LLM-generated Text Detection by Mining Token Probability Sequences](https://mlsanity.com/paper/2410.06072/) (2024)
- [Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Review](https://mlsanity.com/paper/2502.19614/) (2025)
- [Detecting LLM-Generated Peer Reviews](https://mlsanity.com/paper/2503.15772/) (2025)
- [A Practical Examination of AI-Generated Text Detectors for Large Language Models](https://mlsanity.com/paper/2412.05139/) (2024)
- [Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews](https://mlsanity.com/paper/2509.13400/) (2025)
- [exHarmony: Authorship and Citations for Benchmarking the Reviewer Assignment Problem](https://mlsanity.com/paper/2502.07683/) (2025)
- [Detecting AI-Generated Content in Academic Peer Reviews](https://mlsanity.com/paper/2602.00319/) (2026)
- [Meet Your New Client: Writing Reports for AI -- Benchmarking Information Loss in Market Research Deliverables](https://mlsanity.com/paper/2508.15817/) (2025)

## Built on by (most cited first, 4 of 4)

- [LivingArena: Do LLMs Know What Other LLMs Don't? Peer-Probing as Scalable Evaluation](https://mlsanity.com/paper/2607.24780/) (2026)
- [Judgment-Grounded Expansion for Peer Review Generation](https://mlsanity.com/paper/2606.23233/) (2026)
- [Judging a Review by its Cover: A Reliability Analysis of LLM-based Peer Review Evaluation Metrics](https://mlsanity.com/paper/2609.23264/) (2026)
- [Peerify: Benchmarking Peer-Review Claim Verification](https://mlsanity.com/paper/2609.25046/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly used in scientific peer review, assisting with drafting, rewriting, expansion, and refinement. However, existing peer-review LLM detection methods largely treat authorship as a binary problem-human vs. AI-without accounting for the hybrid nature of modern review workflows. In practice, evaluative ideas and surface realization may originate from different sources, creating a spectrum of human-AI collaboration. In this work, we introduce PeerPrism, a large-scale benchmark of 20,690 peer reviews explicitly designed to disentangle idea provenance from text provenance. We construct controlled generation regimes spanning fully human, fully synthetic, and multiple hybrid transformations. This design enables systematic evaluation of whether detectors identify the origin of the surface text or the origin of the evaluative reasoning. We benchmark state-of-the-art LLM text detection methods on PeerPrism. While several methods achieve high accuracy on the standard binary task (human vs. fully synthetic), their predictions diverge sharply under hybrid regimes. In particular, when ideas originate from humans but the surface text is AI-generated, detectors frequently disagree and produce contradictory classifications. Accompanied by stylometric and semantic analyses, our results show that current detection methods conflate surface realization with intellectual contribution. Overall, we demonstrate that LLM detection in peer review cannot be reduced to a binary attribution problem. Instead, authorship must be modeled as a multidimensional construct spanning semantic reasoning and stylistic realization. PeerPrism is the first benchmark evaluating human-AI collaboration in these settings. We release all code, data, prompts, and evaluation scripts to facilitate reproducible research at https://github.com/Reviewerly-Inc/PeerPrism.