# Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems

arXiv: https://arxiv.org/abs/2504.11168 · HTML: https://mlsanity.com/paper/2504.11168/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Combating Adversarial Misspellings with Robust Word Recognition](https://mlsanity.com/paper/1905.11268/) (2019)
- [Safeguarding Large Language Models: A Survey](https://mlsanity.com/paper/2406.02622/) (2024)
- [A Complete Survey on LLM-based AI Chatbots](https://mlsanity.com/paper/2406.16937/) (2024)
- [Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection](https://mlsanity.com/paper/2411.01077/) (2024)

## Built on by (most cited first, 26 of 26)

- [Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens](https://mlsanity.com/paper/2505.13775/) (2025)
- [Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs](https://mlsanity.com/paper/2505.04806/) (2025)
- [Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware](https://mlsanity.com/paper/2607.02357/) (2026)
- [Eliciting and Analyzing Emergent Misalignment in State-of-the-Art Large Language Models](https://mlsanity.com/paper/2508.04196/) (2025)
- [Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Trading](https://mlsanity.com/paper/2601.13082/) (2026)
- [A Framework for Rapidly Developing and Deploying Protection Against Large Language Model Attacks](https://mlsanity.com/paper/2509.20639/) (2025)
- [Toward Reliable, Safe, and Secure LLMs for Scientific Applications](https://mlsanity.com/paper/2603.18235/) (2026)
- [From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails](https://mlsanity.com/paper/2606.14517/) (2026)
- [Behind the Refusal: Determining Guardrail Activation via Behavioral Monitoring](https://mlsanity.com/paper/2607.02121/) (2026)
- [Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models](https://mlsanity.com/paper/2609.19472/) (2026)
- [Information-Dense Reasoning for Efficient and Auditable Security Alert Triage](https://mlsanity.com/paper/2512.08169/) (2025)
- [Language Model Agents Under Attack: A Cross Model-Benchmark of Profit-Seeking Behaviors in Customer Service](https://mlsanity.com/paper/2512.24415/) (2025)
- [WiFiPenTester: Advancing Wireless Ethical Hacking with Governed GenAI](https://mlsanity.com/paper/2601.23092/) (2026)
- [AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue](https://mlsanity.com/paper/2602.17443/) (2026)
- [Beyond Pattern Matching: Seven Cross-Domain Techniques for Prompt Injection Detection](https://mlsanity.com/paper/2604.18248/) (2026)
- [Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs](https://mlsanity.com/paper/2605.06669/) (2026)
- [PsychoPass: Geometric Profiling of Multi-Turn Adversarial LLM Conversations](https://mlsanity.com/paper/2606.03136/) (2026)
- [Hybrid Adversarial Defence for Natural Language Understanding Tasks](https://mlsanity.com/paper/2606.04612/) (2026)
- [Short paper: Models in the dark -- Rectification and erasure under GDPR in ML supply chains](https://mlsanity.com/paper/2606.05946/) (2026)
- [kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail](https://mlsanity.com/paper/2607.02072/) (2026)
- [Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates](https://mlsanity.com/paper/2607.28959/) (2026)
- [Rethinking Agent Security as a Networking Problem](https://mlsanity.com/paper/2608.12172/) (2026)
- [Truth Lies Deep: Countering Semantic Camouflage via Latent Intent Verification](https://mlsanity.com/paper/2608.20378/) (2026)
- [IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks](https://mlsanity.com/paper/2609.03781/) (2026)
- [Decoding Guardrails: XAI-Guided Perturbation Analysis of Prompt Injection Detection](https://mlsanity.com/paper/2609.24801/) (2026)
- [RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents](https://mlsanity.com/paper/2610.06401/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) guardrail systems are designed to protect against prompt injection and jailbreak attacks. However, they remain vulnerable to evasion techniques. We demonstrate two approaches for bypassing LLM prompt injection and jailbreak detection systems via traditional character injection methods and algorithmic Adversarial Machine Learning (AML) evasion techniques. Through testing against six prominent protection systems, including Microsoft's Azure Prompt Shield and Meta's Prompt Guard, we show that both methods can be used to evade detection while maintaining adversarial utility achieving in some instances up to 100% evasion success. Furthermore, we demonstrate that adversaries can enhance Attack Success Rates (ASR) against black-box targets by leveraging word importance ranking computed by offline white-box models. Our findings reveal vulnerabilities within current LLM protection mechanisms and highlight the need for more robust guardrail systems.