mlsanityData to 7 Oct 2026

"Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers

arXiv 2511.01287 ↗First posted 3 Nov 2025cs.CL cs.CRcited by 3

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

With the rapid advancement of AI models, their deployment across diverse tasks has become increasingly widespread. A notable emerging application is leveraging AI models to assist in reviewing scientific papers. However, recent reports have revealed that some papers contain hidden, injected prompts designed to manipulate AI reviewers into providing overly favorable evaluations. In this work, we present an early systematic investigation into this emerging threat. We propose two classes of attacks: (1) static attack, which employs a fixed injection prompt, and (2) iterative attack, which optimizes the injection prompt against a simulated reviewer model to maximize its effectiveness. Both attacks achieve striking performance, frequently inducing full evaluation scores when targeting frontier AI reviewers. Furthermore, we show that these attacks are robust across various settings. To counter this threat, we explore a simple detection-based defense. While it substantially reduces the attack success rate, we demonstrate that an adaptive attacker can partially circumvent this defense. Our findings underscore the need for greater attention and rigorous safeguards against prompt-injection threats in AI-assisted peer review.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 20 of its 31 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

20 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Hidden Prompts in Manuscripts Exploit AI-Assisted Peer Review8 Jul
  2. 2023Prompt Injection attack against LLM-integrated Applications8 Jun

Other references · 18

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  3. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  4. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  5. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  6. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  7. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  8. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 197
  9. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
  10. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
  11. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  12. 2025Large language models for automated scholarly paper review: A surveycited by 59
6 more
  1. 2024Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Reviewcited by 49
  2. 2025Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025cited by 40
  3. 2025TreeReview: A Dynamic Tree of Questions Framework for Deep and Efficient LLM-based Scientific Peer Reviewcited by 31
  4. 2025Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviewscited by 23
  5. 2025ReviewEval: An Evaluation Framework for AI-Generated Reviewscited by 22
  6. 2024PeerArg: Argumentative Peer Review with LLMscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisionscited by 1
  2. 2026The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishingcited by 1
  3. 2026Intelligence Is Not the Bottleneck: Validating an LLM First-Pass Manuscript Score Against Peer-Review Outcomescited by 0