mlsanityData to 14 Oct 2026

Machine Learning for Detection and Analysis of Novel LLM Jailbreaks

arXiv 2510.01644 ↗First posted 2 Oct 2025cs.CL cs.AI cs.CYcited by 1

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) suffer from a range of vulnerabilities that allow malicious users to solicit undesirable responses through manipulation of the input text. These so-called jailbreak prompts are designed to trick the LLM into circumventing the safety guardrails put in place to keep responses acceptable to the developer's policies. In this study, we analyse the ability of different machine learning models to distinguish jailbreak prompts from genuine uses, including looking at our ability to identify jailbreaks that use previously unseen strategies. Our results indicate that using current datasets the best performance is achieved by fine tuning a Bidirectional Encoder Representations from Transformers (BERT) model end-to-end for identifying jailbreaks. We visualise the keywords that distinguish jailbreak from genuine prompts and conclude that explicit reflexivity in prompt structure could be a signal of jailbreak intention.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 34 of its 50 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

34 of 50 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models13 Jun
  2. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks2 Mar
  3. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
  4. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models21 Feb
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023Multilingual Jailbreak Challenges in Large Language Models10 Oct
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  10. 2023Detecting Language Model Attacks with Perplexity27 Aug
  11. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
3 more
  1. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul
  2. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  3. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  6. 2024A Survey on Large Language Models for Code Generationcited by 658
  7. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  8. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  9. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 328
  10. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 274
  11. 2023Pretraining Language Models with Human Preferencescited by 247
  12. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
7 more
  1. 2024Datasets for Large Language Models: A Comprehensive Surveycited by 71
  2. 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
  3. 2020Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis?cited by 27
  4. 2024Schema Matching with Large Language Models: an Experimental Studycited by 20
  5. 2022Semantic and sentiment analysis of selected Bhagavad Gita translations using BERT-based language frameworkcited by 12
  6. 2022Artificial intelligence for topic modelling in Hindu philosophy: mapping themes between the Upanishads and the Bhagavad Gitacited by 6
  7. 2024Large Language User Interfaces: Voice Interactive User Interfaces powered by LLMscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Detecting Jailbreak Attempts in Clinical Training LLMs Through Automated Linguistic Feature Extractioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Improved Large Language Model Jailbreak Detection via Pretrained Embeddings
  2. 2025A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models
  3. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
  4. 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models
  5. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
  6. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation
  7. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily
  8. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis