# Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses

arXiv: https://arxiv.org/abs/2504.02080 · HTML: https://mlsanity.com/paper/2504.02080/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 33 of its 57 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 30 of 33)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher](https://mlsanity.com/paper/2308.06463/) (2023)
- [A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily](https://mlsanity.com/paper/2311.08268/) (2023)
- [LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked](https://mlsanity.com/paper/2308.07308/) (2023)
- [ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs](https://mlsanity.com/paper/2402.11753/) (2024)
- [Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization](https://mlsanity.com/paper/2311.09096/) (2023)
- [Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2401.17263/) (2024)
- [Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast](https://mlsanity.com/paper/2402.08567/) (2024)
- [Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing](https://mlsanity.com/paper/2402.16192/) (2024)
- [Defending LLMs against Jailbreaking Attacks via Backtranslation](https://mlsanity.com/paper/2402.16459/) (2024)
- [Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses](https://mlsanity.com/paper/2406.01288/) (2024)
- [FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models](https://mlsanity.com/paper/2309.05274/) (2023)
- [Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters](https://mlsanity.com/paper/2405.20413/) (2024)
- [Best Practices for Text Annotation with Large Language Models](https://mlsanity.com/paper/2402.05129/) (2024)
- [PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition](https://mlsanity.com/paper/2405.07932/) (2024)

## Built on by (most cited first, 8 of 8)

- [Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants](https://mlsanity.com/paper/2512.15712/) (2025)
- [Sentinel: SOTA model to protect against prompt injections](https://mlsanity.com/paper/2506.05446/) (2025)
- [The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Models](https://mlsanity.com/paper/2505.12287/) (2025)
- [Architecture Matters for Multi-Agent Security](https://mlsanity.com/paper/2604.23459/) (2026)
- [Countermind: A Multi-Layered Security Architecture for Large Language Models](https://mlsanity.com/paper/2510.11837/) (2025)
- [Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization](https://mlsanity.com/paper/2509.20230/) (2025)
- [Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreaking](https://mlsanity.com/paper/2507.08014/) (2025)
- [Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models](https://mlsanity.com/paper/2606.00150/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly popular, powering a wide range of applications. Their widespread use has sparked concerns, especially through jailbreak attacks that bypass safety measures to produce harmful content. In this paper, we present a comprehensive security analysis of large language models (LLMs), addressing critical research questions on the evolution and determinants of model safety. Specifically, we begin by identifying the most effective techniques for detecting jailbreak attacks. Next, we investigate whether newer versions of LLMs offer improved security compared to their predecessors. We also assess the impact of model size on overall security and explore the potential benefits of integrating multiple defense strategies to enhance the security. Our study evaluates both open-source (e.g., LLaMA and Mistral) and closed-source models (e.g., GPT-4) by employing four state-of-the-art attack techniques and assessing the efficacy of three new defensive approaches.