mlsanityData to 14 Oct 2026

A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models

arXiv 2504.04976 ↗First posted 7 Apr 2025cs.CLcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

The study of large language models (LLMs) is a key area in open-world machine learning. Although LLMs demonstrate remarkable natural language processing capabilities, they also face several challenges, including consistency issues, hallucinations, and jailbreak vulnerabilities. Jailbreaking refers to the crafting of prompts that bypass alignment safeguards, leading to unsafe outputs that compromise the integrity of LLMs. This work specifically focuses on the challenge of jailbreak vulnerabilities and introduces a novel taxonomy of jailbreak attacks grounded in the training domains of LLMs. It characterizes alignment failures through generalization, objectives, and robustness gaps. Our primary contribution is a perspective on jailbreak, framed through the different linguistic domains that emerge during LLM training and alignment. This viewpoint highlights the limitations of existing approaches and enables us to classify jailbreak attacks on the basis of the underlying model deficiencies they exploit. Unlike conventional classifications that categorize attacks based on prompt construction methods (e.g., prompt templating), our approach provides a deeper understanding of LLM behavior. We introduce a taxonomy with four categories -- mismatched generalization, competing objectives, adversarial robustness, and mixed attacks -- offering insights into the fundamental nature of jailbreak vulnerabilities. Finally, we present key lessons derived from this taxonomic study.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 38 of its 51 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

38 of 51 references.

References · 38

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 417
  3. 2018Learning to Drive in a Daycited by 273
  4. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 176
  5. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 151
  6. 2020Exploration-Exploitation in Constrained MDPscited by 149
  7. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  8. 2019Learning Adversarial MDPs with Bandit Feedback and Unknown Transitioncited by 85
  9. 2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 80
  10. 2019Online Convex Optimization in Adversarial Markov Decision Processescited by 79
  11. 2019Safe Policies for Reinforcement Learning via Primal-Dual Methodscited by 79
  12. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
26 more
  1. 2020Constrained Upper Confidence Reinforcement Learningcited by 46
  2. 2020Upper Confidence Primal-Dual Reinforcement Learning for CMDP with Adversarial Losscited by 46
  3. 2020Inverse Constrained Reinforcement Learningcited by 46
  4. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 44
  5. 2019Tight Regret Bounds for Model-Based Reinforcement Learning with Greedy Policiescited by 39
  6. 2020Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimizationcited by 35
  7. 2021DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learningcited by 35
  8. 2021A Dual Approach to Constrained Markov Decision Processes with Entropy Regularizationcited by 34
  9. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 28
  10. 2022Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPscited by 27
  11. 2024Truly No-Regret Learning in Constrained MDPscited by 23
  12. 2023Provably Efficient Model-Free Algorithms for Non-stationary CMDPscited by 21
  13. 2021Faster Algorithm and Sharper Analysis for Constrained Markov Decision Processcited by 19
  14. 2023Learning Safety Constraints from Demonstrations with Unknown Rewardscited by 19
  15. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 18
  16. 2017Online Learning in Weakly Coupled Markov Decision Processes: A Convergence Time Studycited by 14
  17. 2020Learning for Dose Allocation in Adaptive Clinical Trials with Safety Constraintscited by 13
  18. 2024Constraint-Adaptive Policy Switching for Offline Safe Reinforcement Learningcited by 13
  19. 2024Optimal Strong Regret and Violation in Constrained MDPs via Policy Optimizationcited by 12
  20. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challengescited by 10
  21. 2023Efficient Rate Optimal Regret for Adversarial Contextual MDPs Using Online Function Approximationcited by 9
  22. 2024Markov Persuasion Processes: Learning to Persuade from Scratchcited by 8
  23. 2020Provably Efficient Model-Free Algorithm for MDPs with Peak Constraintscited by 5
  24. 2022Learning Optimal Dynamic Treatment Regimens Subject to Stagewise Risk Controlscited by 3
  25. 2024Flipping-based Policy for Chance-Constrained Markov Decision Processescited by 3
  26. 2024Information Design with Unknown Priorcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
  2. 2024Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings
  3. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  4. 2025Formalization Driven LLM Prompt Jailbreaking via Reinforcement Learning
  5. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective
  6. 2025Machine Learning for Detection and Analysis of Novel LLM Jailbreaks
  7. 2026SoK: Robustness in Large Language Models against Jailbreak Attacks
  8. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?