mlsanityData to 7 Oct 2026

Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs

arXiv 2404.14461 ↗First posted 22 Apr 2024cs.CL cs.AI cs.CRcited by 27

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models are aligned to be safe, preventing users from generating harmful content like misinformation or instructions for illegal activities. However, previous work has shown that the alignment process is vulnerable to poisoning attacks. Adversaries can manipulate the safety training data to inject backdoors that act like a universal sudo command: adding the backdoor string to any prompt enables harmful responses from models that, otherwise, behave safely. Our competition, co-located at IEEE SaTML 2024, challenged participants to find universal backdoors in several large language models. This report summarizes the key findings and promising ideas for future research.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 18 of its 22 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

18 of 22 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia8 Feb
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  3. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 15

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  5. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  6. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  7. 2023Poisoning Language Models During Instruction Tuningcited by 254
  8. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  9. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  10. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  11. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 193
  12. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
3 more
  1. 2021Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Modelscited by 96
  2. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPTcited by 70
  3. 2020Humpty Dumpty: Controlling Word Meanings via Corpus Poisoningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 9; 2026: 4.

27 papers.

Sort:

By area:

  1. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  2. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 66
  3. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 50
  4. 2024Persistent Pre-Training Poisoning of LLMscited by 46
  5. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 44
  6. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  7. 2024Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competitioncited by 20
  8. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Modelscited by 15
  9. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 14
  10. 2024Jailbreaking as a Reward Misspecification Problemcited by 11
  11. 2025Learning to Interpret Weight Differences in Language Modelscited by 9
  12. 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
15 more
  1. 2024Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defensecited by 6
  2. 2024Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AIcited by 6
  3. 2024AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignmentcited by 5
  4. 2025LLM Misalignment via Adversarial RLHF Platformscited by 5
  5. 2024Neutralizing Backdoors through Information Conflicts for Large Language Modelscited by 4
  6. 2024Model-Editing-Based Jailbreak against Safety-aligned Large Language Modelscited by 4
  7. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  8. 2024Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuitscited by 3
  9. 2026Watermarking LLM Agent Trajectoriescited by 3
  10. 2024Injecting Undetectable Backdoors in Obfuscated Neural Networks and Language Modelscited by 2
  11. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  12. 2025ContextBench: Modifying Contexts for Targeted Latent Activationcited by 0
  13. 2026Critical-CoT: A Robust Defense Framework against Reasoning-Level Backdoor Attacks in Large Language Modelscited by 0
  14. 2026Defense Against LLM Backdoors using Critical Neuron Isolation Pruningcited by 0
  15. 2026Detecting Hidden Behaviors in LLMs via Activation-matched Finetuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Lifelong Safety Alignment for Language Models
  2. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM
  3. 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
  4. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States
  5. 2025TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts
  6. 2024EnJa: Ensemble Jailbreak on Large Language Models
  7. 2024Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings
  8. 2026SecureBreak -- A dataset towards safe and secure models