mlsanityData to 7 Oct 2026

Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction

arXiv 2402.18104 ↗First posted 28 Feb 2024cs.CR cs.AIcited by 95

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

In recent years, large language models (LLMs) have demonstrated notable success across various tasks, but the trustworthiness of LLMs is still an open problem. One specific threat is the potential to generate toxic or harmful responses. Attackers can craft adversarial prompts that induce harmful responses from LLMs. In this work, we pioneer a theoretical foundation in LLMs security by identifying bias vulnerabilities within the safety fine-tuning and design a black-box jailbreak method named DRA (Disguise and Reconstruction Attack), which conceals harmful instructions through disguise and prompts the model to reconstruct the original harmful instruction within its completion. We evaluate DRA across various open-source and closed-source models, showcasing state-of-the-art jailbreak success rates and attack efficiency. Notably, DRA boasts a 91.1% attack success rate on OpenAI GPT-4 chatbot.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 38 of its 57 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

38 of 57 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  3. 2023Multilingual Jailbreak Challenges in Large Language Models10 Oct
  4. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  5. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  6. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  7. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  9. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul
  10. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023Mistral 7Bcited by 3,065
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  11. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  12. 2024Mixtral of Expertscited by 1,785
16 more
  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  4. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  5. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  6. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  7. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  8. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  9. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  10. 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 92
  11. 2024LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoningcited by 57
  12. 2023Beyond the Safeguards: Exploring the Security Risks of ChatGPTcited by 36
  13. 2023Demystifying RCE Vulnerabilities in LLM-Integrated Appscited by 35
  14. 2023Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Frameworkcited by 29
  15. 2023Demonstration of InsightPilot: An LLM-Empowered Automated Data Exploration Systemcited by 20
  16. 2022Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Costcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 26; 2025: 48; 2026: 21.

95 papers.

Sort:

By area:

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  2. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 94
  3. 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agentscited by 81
  4. 2024Safeguarding Large Language Models: A Surveycited by 73
  5. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  6. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  7. 2024Imprompter: Tricking LLM Agents into Improper Tool Usecited by 66
  8. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  9. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 54
  10. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  11. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  12. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
38 more
  1. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 39
  2. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  3. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreakingcited by 31
  4. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 29
  5. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  6. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  7. 2024Adversarial Tuning: Defending Against Jailbreak Attacks for LLMscited by 23
  8. 2024h4rm3l: A language for Composable Jailbreak Attack Synthesiscited by 23
  9. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  10. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  11. 2025AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basincited by 18
  12. 2024Jailbreaking Large Language Models with Symbolic Mathematicscited by 14
  13. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 13
  14. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 11
  15. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
  16. 2026Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Searchcited by 10
  17. 2024Diversity Helps Jailbreak Large Language Modelscited by 9
  18. 2025GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methodscited by 9
  19. 2025Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversioncited by 9
  20. 2025LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challengescited by 9
  21. 2024Jailbreaking? One Step Is Enough!cited by 8
  22. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  23. 2024Reasoning as a Weapon: Adaptive Dual-Path Jailbreak Attack on Large Language Modelscited by 7
  24. 2025Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defensecited by 7
  25. 2026Understanding and Preserving Safety in Fine-Tuned LLMscited by 7
  26. 2025From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMscited by 6
  27. 2025Hiding in Plain Sight: A Steganographic Approach to Stealthy LLM Jailbreakscited by 6
  28. 2025ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistantscited by 6
  29. 2026HoneyTrap: Deceiving Large Language Model Attackers to Honeypot Traps with Resilient Multi-Agent Defensecited by 6
  30. 2026Knowledge-Driven Multi-Turn Jailbreaking on Large Language Modelscited by 6
  31. 2024AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimizationcited by 5
  32. 2025Bleeding Pathways: Vanishing Discriminability in LLM Hidden States Fuels Jailbreak Attackscited by 5
  33. 2025Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processingcited by 5
  34. 2025Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Modelscited by 5
  35. 2025MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?cited by 5
  36. 2025Universally Unfiltered and Unseen:Input-Agnostic Multimodal Jailbreaks against Text-to-Image Model Safeguardscited by 5
  37. 2025Harmful Prompt Laundering: Jailbreaking LLMs with Abductive Styles and Symbolic Encodingcited by 5
  38. 2025When Reject Turns into Accept: Quantifying the Vulnerability of LLM-Based Scientific Reviewers to Indirect Prompt Injectioncited by 5

50 most cited of 95 All 95 (JSON)