Preventing the Collapse of Peer Review Requires Verification-First AI
Not on the map. Why?
Abstract (arXiv)
This paper argues that AI-assisted peer review should be verification-first rather than review-mimicking. We propose truth-coupling, i.e. how tightly venue scores track latent scientific truth, as the right objective for review tools. We formalize two forces that drive a phase transition toward proxy-sovereign evaluation: verification pressure, when claims outpace verification capacity, and signal shrinkage, when real improvements become hard to separate from noise. In a minimal model that mixes occasional high-fidelity checks with frequent proxy judgment, we derive an explicit coupling law and an incentive-collapse condition under which rational effort shifts from truth-seeking to proxy optimization, even when current decisions still appear reliable. These results motivate actions for tool builders and program chairs: deploy AI as an adversarial auditor that generates auditable verification artifacts and expands effective verification bandwidth, rather than as a score predictor that amplifies claim inflation.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 13 of its 33 references that are in the map. First posted on arXiv 23 Jan 2026.
Builds on
13 of 33 references.
References · 13
- 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
- 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 135
- 2018Categorizing Variants of Goodhart's Lawcited by 132
- 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
- 2024Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experimentcited by 15
- 2025Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedbackcited by 12
- 2025FLAWS: A Benchmark for Error Identification and Localization in Scientific Paperscited by 12
- 2024Problem Solving Through Human-AI Preference-Based Cooperationcited by 11
- 2024Evaluating the Predictive Capacity of ChatGPT for Academic Peer Review Outcomes Across Multiple Platformscited by 11
- 2025Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Frameworkcited by 10
- 2026SciCoQA: Quality Assurance for Scientific Paper--Code Alignmentcited by 9
- 2024STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyondcited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.