# When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Review

arXiv: https://arxiv.org/abs/2509.09912 · HTML: https://mlsanity.com/paper/2509.09912/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 31 of its 74 references that are in the map.
- First posted on arXiv 12 Sep 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 30 of 31)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [Knowledge Editing for Large Language Models: A Survey](https://mlsanity.com/paper/2310.16218/) (2023)
- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process](https://mlsanity.com/paper/2503.08569/) (2025)
- [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024)
- [The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates](https://mlsanity.com/paper/2405.02150/) (2024)
- [Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making](https://mlsanity.com/paper/1802.01029/) (2018)
- [A Bibliometric Review of Large Language Models Research from 2017 to 2023](https://mlsanity.com/paper/2304.02020/) (2023)
- [Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review](https://mlsanity.com/paper/2412.01708/) (2024)
- [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021)
- [Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025](https://mlsanity.com/paper/2504.09737/) (2025)
- [AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews](https://mlsanity.com/paper/2408.10365/) (2024)
- [Defense Against Prompt Injection Attack by Leveraging Attack Techniques](https://mlsanity.com/paper/2411.00459/) (2024)
- [Improving Human-AI Collaboration With Descriptions of AI Behavior](https://mlsanity.com/paper/2301.06937/) (2023)
- [Embedding-based classifiers can detect prompt injection attacks](https://mlsanity.com/paper/2410.22284/) (2024)
- [FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks](https://mlsanity.com/paper/2410.21492/) (2024)
- [Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Review](https://mlsanity.com/paper/2410.03019/) (2024)
- [Harmful Speech Detection by Language Models Exhibits Gender-Queer Dialect Bias](https://mlsanity.com/paper/2406.00020/) (2024)
- [Position: Adversarial ML for LLMs Is Not Making Any Progress](https://mlsanity.com/paper/2502.02260/) (2025)
- [Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations](https://mlsanity.com/paper/2503.01623/) (2025)
- [Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models](https://mlsanity.com/paper/2505.16957/) (2025)
- [Comparison of Topic Modelling Approaches in the Banking Context](https://mlsanity.com/paper/2402.03176/) (2024)

## Built on by (most cited first, 19 of 19)

- [The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis](https://mlsanity.com/paper/2602.10453/) (2026)
- [Stop Automating Peer Review Without Rigorous Evaluation](https://mlsanity.com/paper/2605.03202/) (2026)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [LLM-REVal: Can We Trust LLM Reviewers Yet?](https://mlsanity.com/paper/2510.12367/) (2025)
- [Paraphrasing Adversarial Attack on LLM-as-a-Reviewer](https://mlsanity.com/paper/2601.06884/) (2026)
- [LLM-as-a-Reviewer: Benchmarking Their Ability, Divergence, and Prompt Injection Resistance as Paper Reviewers](https://mlsanity.com/paper/2605.25415/) (2026)
- [PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Review](https://mlsanity.com/paper/2601.19916/) (2026)
- [Do LLMs Favor LLMs? Quantifying Interaction Effects in Peer Review](https://mlsanity.com/paper/2601.20920/) (2026)
- [How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review](https://mlsanity.com/paper/2608.08975/) (2026)
- [From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation](https://mlsanity.com/paper/2603.17588/) (2026)
- [FactReview: Evidence-Grounded Peer Review with Execution-Based Claim Verification](https://mlsanity.com/paper/2604.04074/) (2026)
- [CoCoReviewBench: A Completeness- and Correctness-Oriented Benchmark for AI Reviewers](https://mlsanity.com/paper/2605.07905/) (2026)
- [No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions](https://mlsanity.com/paper/2606.13044/) (2026)
- [RubricReviewer: From Direct Critique to Objective and Comprehensive Rubric-Driven Peer Review](https://mlsanity.com/paper/2608.00005/) (2026)
- [Decoupling Scores and Text: The Politeness Principle in Peer Review](https://mlsanity.com/paper/2604.14162/) (2026)
- [LLM-Based Scientific Peer Review: Methods, Benchmarks, and Reliability Challenges](https://mlsanity.com/paper/2606.25057/) (2026)
- [The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing](https://mlsanity.com/paper/2609.07713/) (2026)
- [ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation](https://mlsanity.com/paper/2609.09076/) (2026)
- [A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review](https://mlsanity.com/paper/2609.39027/) (2026)

## Abstract (arXiv)

Peer review is the cornerstone of academic publishing, yet the process is increasingly strained by rising submission volumes, reviewer overload, and expertise mismatches. Large language models (LLMs) are now being used as "reviewer aids," raising concerns about their fairness, consistency, and robustness against indirect prompt injection attacks. This paper presents a systematic evaluation of LLMs as academic reviewers. Using a curated dataset of 1,441 papers from ICLR 2023 and NeurIPS 2022, we evaluate GPT-5-mini against human reviewers across ratings, strengths, and weaknesses. The evaluation employs structured prompting with reference paper calibration, topic modeling, and similarity analysis to compare review content. We further embed covert instructions into PDF submissions to assess LLMs' susceptibility to prompt injection. Our findings show that LLMs consistently inflate ratings for weaker papers while aligning more closely with human judgments on stronger contributions. Moreover, while overarching malicious prompts induce only minor shifts in topical focus, explicitly field-specific instructions successfully manipulate specific aspects of LLM-generated reviews. This study underscores both the promises and perils of integrating LLMs into peer review and points to the importance of designing safeguards that ensure integrity and trust in future review processes.