# Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs

arXiv: https://arxiv.org/abs/2404.14461 · HTML: https://mlsanity.com/paper/2404.14461/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 22 Apr 2024.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 18 of 18)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Poisoning Language Models During Instruction Tuning](https://mlsanity.com/paper/2305.00944/) (2023)
- [The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning](https://mlsanity.com/paper/2312.01552/) (2023)
- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications](https://mlsanity.com/paper/2402.05162/) (2024)
- [Foundational Challenges in Assuring Alignment and Safety of Large Language Models](https://mlsanity.com/paper/2404.09932/) (2024)
- [Universal Jailbreak Backdoors from Poisoned Human Feedback](https://mlsanity.com/paper/2311.14455/) (2023)
- [Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models](https://mlsanity.com/paper/2103.15543/) (2021)
- [BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT](https://mlsanity.com/paper/2304.12298/) (2023)
- [Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning](https://mlsanity.com/paper/2001.04935/) (2020)
- [Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia](https://mlsanity.com/paper/2402.05467/) (2024)

## Built on by (most cited first, 28 of 28)

- [Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs](https://mlsanity.com/paper/2407.15549/) (2024)
- [Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples](https://mlsanity.com/paper/2510.07192/) (2025)
- [BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models](https://mlsanity.com/paper/2406.17092/) (2024)
- [BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models](https://mlsanity.com/paper/2408.12798/) (2024)
- [Persistent Pre-Training Poisoning of LLMs](https://mlsanity.com/paper/2410.13722/) (2024)
- [A Survey of Recent Backdoor Attacks and Defenses in Large Language Models](https://mlsanity.com/paper/2406.06852/) (2024)
- [Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition](https://mlsanity.com/paper/2406.07954/) (2024)
- [Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models](https://mlsanity.com/paper/2506.16447/) (2025)
- [Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges](https://mlsanity.com/paper/2507.19672/) (2025)
- [Jailbreaking as a Reward Misspecification Problem](https://mlsanity.com/paper/2406.14393/) (2024)
- [Learning to Interpret Weight Differences in Language Models](https://mlsanity.com/paper/2510.05092/) (2025)
- [Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense](https://mlsanity.com/paper/2410.09838/) (2024)
- [Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs](https://mlsanity.com/paper/2508.00161/) (2025)
- [Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI](https://mlsanity.com/paper/2410.15467/) (2024)
- [AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment](https://mlsanity.com/paper/2410.11283/) (2024)
- [Neutralizing Backdoors through Information Conflicts for Large Language Models](https://mlsanity.com/paper/2411.18280/) (2024)
- [LLM Misalignment via Adversarial RLHF Platforms](https://mlsanity.com/paper/2503.03039/) (2025)
- [SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses](https://mlsanity.com/paper/2510.15476/) (2025)
- [Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuits](https://mlsanity.com/paper/2406.02619/) (2024)
- [Model-Editing-Based Jailbreak against Safety-aligned Large Language Models](https://mlsanity.com/paper/2412.08201/) (2024)
- [Watermarking LLM Agent Trajectories](https://mlsanity.com/paper/2602.18700/) (2026)
- [Injecting Undetectable Backdoors in Obfuscated Neural Networks and Language Models](https://mlsanity.com/paper/2406.05660/) (2024)
- [Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation](https://mlsanity.com/paper/2508.10404/) (2025)
- [Critical-CoT: A Robust Defense Framework against Reasoning-Level Backdoor Attacks in Large Language Models](https://mlsanity.com/paper/2604.10681/) (2026)
- [ContextBench: Modifying Contexts for Targeted Latent Activation](https://mlsanity.com/paper/2506.15735/) (2025)
- [The Persistent Vulnerability of Aligned AI Systems](https://mlsanity.com/paper/2604.00324/) (2026)
- [Defense Against LLM Backdoors using Critical Neuron Isolation Pruning](https://mlsanity.com/paper/2607.19894/) (2026)
- [Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning](https://mlsanity.com/paper/2609.00351/) (2026)

## Abstract (arXiv)

Large language models are aligned to be safe, preventing users from generating harmful content like misinformation or instructions for illegal activities. However, previous work has shown that the alignment process is vulnerable to poisoning attacks. Adversaries can manipulate the safety training data to inject backdoors that act like a universal sudo command: adding the backdoor string to any prompt enables harmful responses from models that, otherwise, behave safely. Our competition, co-located at IEEE SaTML 2024, challenged participants to find universal backdoors in several large language models. This report summarizes the key findings and promising ideas for future research.