mlsanityData to 7 Oct 2026

When Reject Turns into Accept: Quantifying the Vulnerability of LLM-Based Scientific Reviewers to Indirect Prompt Injection

arXiv 2512.10449 ↗First posted 11 Dec 2025cs.AI cs.CL cs.CRcited by 5

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Driven by surging submission volumes, scientific peer review has catalyzed two parallel trends: individual over-reliance on LLMs and institutional AI-powered assessment systems. This study investigates the robustness of "LLM-as-a-Judge" systems to adversarial PDF manipulation via invisible text injections and layout aware encoding attacks. We specifically target the distinct incentive of flipping "Reject" decisions to "Accept," a vulnerability that fundamentally compromises scientific integrity. To measure this, we introduce the Weighted Adversarial Vulnerability Score (WAVS), a novel metric that quantifies susceptibility by weighting score inflation against the severity of decision shifts relative to ground truth. We adapt 15 domain-specific attack strategies, ranging from semantic persuasion to cognitive obfuscation, and evaluate them across 13 diverse language models (including GPT-5 and DeepSeek) using a curated dataset of 200 official and real-world accepted and rejected submissions (e.g., ICLR OpenReview). Our results demonstrate that obfuscation techniques like "Maximum Mark Magyk" and "Symbolic Masking & Context Redirection" successfully manipulate scores, achieving decision flip rates of up to 86.26% in open-source models, while exposing distinct "reasoning traps" in proprietary systems. We release our complete dataset and injection framework to facilitate further research on the topic (https://anonymous.4open.sciencer/llm-jailbreak-FC9E/).

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 18 of its 44 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

18 of 44 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Prompt Injection Attacks on LLM Generated Reviews of Scientific Publications12 Sep
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 16

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  3. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  4. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  5. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  6. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  7. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 95
  8. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 92
  9. 2024OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviewscited by 63
  10. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Cluescited by 57
  11. 2025Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategycited by 37
  12. 2025ReviewEval: An Evaluation Framework for AI-Generated Reviewscited by 22
4 more
  1. 2024Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Reviewcited by 19
  2. 2024Virtual Context: Enhancing Jailbreak Attacks with Special Token Injectioncited by 16
  3. 2025BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judgecited by 13
  4. 2025Investigating the Vulnerability of LLM-as-a-Judge Architectures to Prompt-Injection Attackscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Paraphrasing Adversarial Attack on LLM-as-a-Reviewercited by 6
  2. 2026Machine-Assisted Grading of Nationwide School-Leaving Essay Exams with LLMs and Statistical NLPcited by 2
  3. 2026IntraGuard: Committee-Side Defenses Against Review Outsourcing to Commercial Chatbotscited by 2
  4. 2026The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishingcited by 1
  5. 2026Breaking the Illusion of Review Reliability under Static Evaluation: SCOPE Fuzzing for LLM-based Scientific Reviewerscited by 0