mlsanityData to 14 Oct 2026

Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems

arXiv 2504.11168 ↗First posted 15 Apr 2025cs.CR cs.AI cs.LGcited by 26

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) guardrail systems are designed to protect against prompt injection and jailbreak attacks. However, they remain vulnerable to evasion techniques. We demonstrate two approaches for bypassing LLM prompt injection and jailbreak detection systems via traditional character injection methods and algorithmic Adversarial Machine Learning (AML) evasion techniques. Through testing against six prominent protection systems, including Microsoft's Azure Prompt Shield and Meta's Prompt Guard, we show that both methods can be used to evade detection while maintaining adversarial utility achieving in some instances up to 100% evasion success. Furthermore, we demonstrate that adversaries can enhance Attack Success Rates (ASR) against black-box targets by leveraging word importance ranking computed by offline white-box models. Our findings reveal vulnerabilities within current LLM protection mechanisms and highlight the need for more robust guardrail systems.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 15 of its 30 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

15 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection1 Nov

Other references · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  4. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  5. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  6. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  7. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  8. 2018Generating Natural Language Adversarial Examplescited by 424
  9. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  10. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  11. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  12. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 163
2 more
  1. 2024Safeguarding Large Language Models: A Surveycited by 78
  2. 2024A Complete Survey on LLM-based AI Chatbotscited by 71

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 20.

26 papers.

Sort:

By area:

  1. 2025Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokenscited by 41
  2. 2025Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMscited by 34
  3. 2026Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malwarecited by 10
  4. 2025Eliciting and Analyzing Emergent Misalignment in State-of-the-Art Large Language Modelscited by 4
  5. 2026Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Tradingcited by 4
  6. 2025A Framework for Rapidly Developing and Deploying Protection Against Large Language Model Attackscited by 3
  7. 2026Toward Reliable, Safe, and Secure LLMs for Scientific Applicationscited by 1
  8. 2026From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrailscited by 1
  9. 2026Behind the Refusal: Determining Guardrail Activation via Behavioral Monitoringcited by 1
  10. 2026Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Modelscited by 1
  11. 2025Information-Dense Reasoning for Efficient and Auditable Security Alert Triagecited by 0
  12. 2025Language Model Agents Under Attack: A Cross Model-Benchmark of Profit-Seeking Behaviors in Customer Servicecited by 0
14 more
  1. 2026WiFiPenTester: Advancing Wireless Ethical Hacking with Governed GenAIcited by 0
  2. 2026AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialoguecited by 0
  3. 2026Beyond Pattern Matching: Seven Cross-Domain Techniques for Prompt Injection Detectioncited by 0
  4. 2026Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offscited by 0
  5. 2026PsychoPass: Geometric Profiling of Multi-Turn Adversarial LLM Conversationscited by 0
  6. 2026Hybrid Adversarial Defence for Natural Language Understanding Taskscited by 0
  7. 2026Short paper: Models in the dark -- Rectification and erasure under GDPR in ML supply chainscited by 0
  8. 2026kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrailcited by 0
  9. 2026Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogatescited by 0
  10. 2026Rethinking Agent Security as a Networking Problemcited by 0
  11. 2026Truth Lies Deep: Countering Semantic Camouflage via Latent Intent Verificationcited by 0
  12. 2026IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attackscited by 0
  13. 2026Decoding Guardrails: XAI-Guided Perturbation Analysis of Prompt Injection Detectioncited by 0
  14. 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMs
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  3. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Models
  4. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
  5. 2025SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
  6. 2024MoJE: Mixture of Jailbreak Experts, Naive Tabular Classifiers as Guard for Prompt Attacks
  7. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models
  8. 2026A Systematic Literature Review on LLM Defenses Against Prompt Injection and Jailbreaking: Expanding NIST Taxonomy