mlsanityData to 7 Oct 2026

Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injections

arXiv 2504.18333 ↗First posted 25 Apr 2025cs.CR cs.CLcited by 5

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

LLM as judge systems used to assess text quality code correctness and argument strength are vulnerable to prompt injection attacks. We introduce a framework that separates content author attacks from system prompt attacks and evaluate five models Gemma 3.27B Gemma 3.4B Llama 3.2 3B GPT 4 and Claude 3 Opus on four tasks with various defenses using fifty prompts per condition. Attacks achieved up to seventy three point eight percent success smaller models proved more vulnerable and transferability ranged from fifty point five to sixty two point six percent. Our results contrast with Universal Prompt Injection and AdvPrompter We recommend multi model committees and comparative scoring and release all code and datasets

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 20 of its 34 references that are in the map. First posted on arXiv 25 Apr 2025.

Builds on

20 of 34 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Exploring Vulnerabilities and Protections in Large Language Models: A Survey1 Jun
  2. 2024Automatic and Universal Prompt Injection Attacks against Large Language Models7 Mar
  3. 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb
  4. 2023Prompt Injection attack against LLM-integrated Applications8 Jun

Other references · 16

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  6. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  8. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  9. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  10. 2024Self-Preference Bias in LLM-as-a-Judgecited by 248
  11. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  12. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
4 more
  1. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  2. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  3. 2024Explaining Length Bias in LLM-Based Preference Evaluationscited by 48
  4. 2024Exploring the Robustness of Language Models for Tabular Question Answering via Attention Analysiscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysiscited by 25
  2. 2026A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustnesscited by 20
  3. 2026What Can Be Enforced? A Theory of Certified Runtime Safety for Tool-Using Agentscited by 1
  4. 2026Making Bias Non-Predictive: Training Robust LLM Reasoning via Reinforcement Learningcited by 0
  5. 2026Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerabilitycited by 0