mlsanityData to 7 Oct 2026

Jailbreaking? One Step Is Enough!

arXiv 2412.12621 ↗First posted 17 Dec 2024cs.CLcited by 8

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models (LLMs) excel in various tasks but remain vulnerable to jailbreak attacks, where adversaries manipulate prompts to generate harmful outputs. Examining jailbreak prompts helps uncover the shortcomings of LLMs. However, current jailbreak methods and the target model's defenses are engaged in an independent and adversarial process, resulting in the need for frequent attack iterations and redesigning attacks for different models. To address these gaps, we propose a Reverse Embedded Defense Attack (REDA) mechanism that disguises the attack intention as the "defense". intention against harmful content. Specifically, REDA starts from the target response, guiding the model to embed harmful content within its defensive measures, thereby relegating harmful content to a secondary role and making the model believe it is performing a defensive task. The attacking model considers that it is guiding the target model to deal with harmful content, while the target model thinks it is performing a defensive task, creating an illusion of cooperation between the two. Additionally, to enhance the model's confidence and guidance in "defensive" intentions, we adopt in-context learning (ICL) with a small number of attack examples and construct a corresponding dataset of attack examples. Extensive evaluations demonstrate that the REDA method enables cross-model attacks without the need to redesign attack strategies for different models, enables successful jailbreak in one iteration, and outperforms existing methods on both open-source and closed-source models.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 25 of its 34 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

25 of 34 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction28 Feb
  2. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding14 Feb
  3. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  4. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  7. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  9. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul
  10. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  5. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  6. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  7. 2022Red Teaming Language Models with Language Modelscited by 983
  8. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  9. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  10. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  11. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  12. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
3 more
  1. 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 92
  2. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51
  3. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.