# Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks

arXiv: https://arxiv.org/abs/2506.11113 · HTML: https://mlsanity.com/paper/2506.11113/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Builds on (most cited first, 24 of 24)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications](https://mlsanity.com/paper/1804.09635/) (2018)
- [Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment](https://mlsanity.com/paper/2402.14016/) (2024)
- [ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing](https://mlsanity.com/paper/2306.00622/) (2023)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [Generating Natural Language Attacks in a Hard Label Black Box Setting](https://mlsanity.com/paper/2012.14956/) (2020)
- [AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews](https://mlsanity.com/paper/2408.10365/) (2024)
- [GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study](https://mlsanity.com/paper/2307.05492/) (2023)
- [Design and Analysis of the NIPS 2016 Review Process](https://mlsanity.com/paper/1708.09794/) (2017)
- [Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Review](https://mlsanity.com/paper/2410.03019/) (2024)
- [Automatic generation of reviews of scientific papers](https://mlsanity.com/paper/2010.04147/) (2020)
- [The Memory of Science: Inflation, Myopia, and the Knowledge Network](https://mlsanity.com/paper/1607.05606/) (2016)

## Built on by (most cited first, 16 of 16)

- [The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems](https://mlsanity.com/paper/2509.08713/) (2025)
- [Stop Automating Peer Review Without Rigorous Evaluation](https://mlsanity.com/paper/2605.03202/) (2026)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [LLM-REVal: Can We Trust LLM Reviewers Yet?](https://mlsanity.com/paper/2510.12367/) (2025)
- [Paraphrasing Adversarial Attack on LLM-as-a-Reviewer](https://mlsanity.com/paper/2601.06884/) (2026)
- [SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Prompts](https://mlsanity.com/paper/2604.26506/) (2026)
- [APRES: An Agentic Paper Revision and Evaluation System](https://mlsanity.com/paper/2603.03142/) (2026)
- [Evaluating the Impact of Reviewer Guideline Design on LLM-Based Automated Peer Review](https://mlsanity.com/paper/2607.22553/) (2026)
- [When AI reviews science: Can we trust the referee?](https://mlsanity.com/paper/2604.23593/) (2026)
- [PeeriScope: A Multi-Faceted Framework for Evaluating Peer Review Quality](https://mlsanity.com/paper/2604.24071/) (2026)
- [TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews](https://mlsanity.com/paper/2605.26911/) (2026)
- [Does AI Reviewer See the Full Picture? Attacking and Defending Multimodal Peer Review](https://mlsanity.com/paper/2606.12716/) (2026)
- [The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing](https://mlsanity.com/paper/2609.07713/) (2026)
- [Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extraction](https://mlsanity.com/paper/2604.07964/) (2026)
- [A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review](https://mlsanity.com/paper/2609.39027/) (2026)
- [How Robust Is Multimodal Claim Verification to LLM Rewriting?](https://mlsanity.com/paper/2610.02841/) (2026)

## Abstract (arXiv)

Peer review is essential for maintaining academic quality, but the increasing volume of submissions places a significant burden on reviewers. Large language models (LLMs) offer potential assistance in this process, yet their susceptibility to textual adversarial attacks raises reliability concerns. This paper investigates the robustness of LLMs used as automated reviewers in the presence of such attacks. We focus on three key questions: (1) The effectiveness of LLMs in generating reviews compared to human reviewers. (2) The impact of adversarial attacks on the reliability of LLM-generated reviews. (3) Challenges and potential mitigation strategies for LLM-based review. Our evaluation reveals significant vulnerabilities, as text manipulations can distort LLM assessments. We offer a comprehensive evaluation of LLM performance in automated peer reviewing and analyze its robustness against adversarial attacks. Our findings emphasize the importance of addressing adversarial risks to ensure AI strengthens, rather than compromises, the integrity of scholarly communication.