mlsanityData to 14 Oct 2026

Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses

arXiv 2504.02080 ↗First posted 2 Apr 2025cs.CR cs.AIcited by 8

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) are increasingly popular, powering a wide range of applications. Their widespread use has sparked concerns, especially through jailbreak attacks that bypass safety measures to produce harmful content. In this paper, we present a comprehensive security analysis of large language models (LLMs), addressing critical research questions on the evolution and determinants of model safety. Specifically, we begin by identifying the most effective techniques for detecting jailbreak attacks. Next, we investigate whether newer versions of LLMs offer improved security compared to their predecessors. We also assess the impact of model size on overall security and explore the potential benefits of integrating multiple defense strategies to enhance the security. Our study evaluates both open-source (e.g., LLaMA and Mistral) and closed-source models (e.g., GPT-4) by employing four state-of-the-art attack techniques and assessing the efficacy of three new defensive approaches.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 33 of its 57 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

33 of 57 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses3 Jun
  2. 2024Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters30 May
  3. 2024PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition13 May
  4. 2024Protecting Your LLMs with Information Bottleneck22 Apr
  5. 2024Defending LLMs against Jailbreaking Attacks via Backtranslation26 Feb
  6. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
  7. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  8. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks30 Jan
  9. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  10. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  11. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  12. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
6 more
  1. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  3. 2023FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models11 Sep
  4. 2023Detecting Language Model Attacks with Perplexity27 Aug
  5. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  6. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 15

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2023Mistral 7Bcited by 3,164
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  8. 2024A Survey on Large Language Models for Code Generationcited by 658
  9. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  10. 2019Text Summarization with Pretrained Encoderscited by 556
  11. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  12. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
3 more
  1. 2024Best Practices for Text Annotation with Large Language Modelscited by 36
  2. 2021Two Contrasting Data Annotation Paradigms for Subjective NLP Taskscited by 12
  3. 2021Thief, Beware of What Get You There: Towards Understanding Model Extraction Attackcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistantscited by 12
  2. 2025Sentinel: SOTA model to protect against prompt injectionscited by 6
  3. 2025The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Modelscited by 3
  4. 2026Architecture Matters for Multi-Agent Securitycited by 3
  5. 2025Countermind: A Multi-Layered Security Architecture for Large Language Modelscited by 2
  6. 2025Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimizationcited by 1
  7. 2025Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreakingcited by 0
  8. 2026Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
  2. 2024A Cross-Language Investigation into Jailbreak Attacks in Large Language Models
  3. 2024Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks
  4. 2025You Can't Eat Your Cake and Have It Too: The Performance Degradation of LLMs with Jailbreak Defense
  5. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
  6. 2026SoK: Robustness in Large Language Models against Jailbreak Attacks
  7. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
  8. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks