mlsanityData to 7 Oct 2026

Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks

arXiv 2506.11113 ↗First posted 8 Jun 2025cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Peer review is essential for maintaining academic quality, but the increasing volume of submissions places a significant burden on reviewers. Large language models (LLMs) offer potential assistance in this process, yet their susceptibility to textual adversarial attacks raises reliability concerns. This paper investigates the robustness of LLMs used as automated reviewers in the presence of such attacks. We focus on three key questions: (1) The effectiveness of LLMs in generating reviews compared to human reviewers. (2) The impact of adversarial attacks on the reliability of LLM-generated reviews. (3) Challenges and potential mitigation strategies for LLM-based review. Our evaluation reveals significant vulnerabilities, as text manipulations can distort LLM assessments. We offer a comprehensive evaluation of LLM performance in automated peer reviewing and analyze its robustness against adversarial attacks. Our findings emphasize the importance of addressing adversarial risks to ensure AI strengthens, rather than compromises, the integrity of scholarly communication.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 24 of its 55 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

24 of 55 references.

References · 24

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  5. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  6. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  7. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  8. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  9. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  10. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 203
  11. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
  12. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 181
12 more
  1. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  2. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 135
  3. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
  4. 2024Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessmentcited by 117
  5. 2020Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databasescited by 75
  6. 2020Generating Natural Language Attacks in a Hard Label Black Box Settingcited by 50
  7. 2024AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviewscited by 42
  8. 2023GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Studycited by 40
  9. 2017Design and Analysis of the NIPS 2016 Review Processcited by 33
  10. 2024Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Reviewcited by 19
  11. 2020Automatic generation of reviews of scientific paperscited by 6
  12. 2016The Memory of Science: Inflation, Myopia, and the Knowledge Networkcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 12.

14 papers.

Sort:

By area:

  1. 2025The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systemscited by 24
  2. 2026Stop Automating Peer Review Without Rigorous Evaluationcited by 12
  3. 2025LLM-REVal: Can We Trust LLM Reviewers Yet?cited by 8
  4. 2026Paraphrasing Adversarial Attack on LLM-as-a-Reviewercited by 6
  5. 2026SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Promptscited by 4
  6. 2026Evaluating the Impact of Reviewer Guideline Design on LLM-Based Automated Peer Reviewcited by 3
  7. 2026When AI reviews science: Can we trust the referee?cited by 2
  8. 2026PeeriScope: A Multi-Faceted Framework for Evaluating Peer Review Qualitycited by 2
  9. 2026APRES: An Agentic Paper Revision and Evaluation Systemcited by 1
  10. 2026TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviewscited by 1
  11. 2026Does AI Reviewer See the Full Picture? Attacking and Defending Multimodal Peer Reviewcited by 1
  12. 2026The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishingcited by 1
2 more
  1. 2026Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extractioncited by 0
  2. 2026A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Reviewcited by 0