# Safety alignment and jailbreak attacks

Area: [LLM alignment / RLHF / preference optimisation](https://mlsanity.com/alignment/index.md). HTML: https://mlsanity.com/alignment/safety-alignment-and-jailbreak-attacks/

## Key facts

- Safety alignment and jailbreak attacks: 687 papers on arXiv since 2019 (data through 7 Oct 2026).
- 203 of its 687 mapped papers (30%) cite “Universal and Transferable Adversarial Attacks on Aligned Language Models” (2023, arXiv 2307.15043).
- 127 of its 687 mapped papers (18%) cite “Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!” (2023, arXiv 2310.03693).
- 30 new papers 17 Sep–7 Oct.

## Most cited here

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023), cited by 203 papers here
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023), cited by 127 papers here
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023), cited by 98 papers here
- [Safety Alignment Should Be Made More Than Just a Few Tokens Deep](https://mlsanity.com/paper/2406.05946/) (2024), cited by 87 papers here
- [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.12773/) (2023), cited by 75 papers here
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023), cited by 57 papers here
- [Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models](https://mlsanity.com/paper/2310.02949/) (2023), cited by 51 papers here
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023), cited by 43 papers here
- [PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference](https://mlsanity.com/paper/2406.15513/) (2024), cited by 41 papers here
- [Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications](https://mlsanity.com/paper/2402.05162/) (2024), cited by 38 papers here

## New (17 Sep–7 Oct)

- [How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defense](https://mlsanity.com/paper/2610.11005/) (2026-10-07)
- [Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models](https://mlsanity.com/paper/2610.10625/) (2026-10-07)
- [SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing](https://mlsanity.com/paper/2610.10345/) (2026-10-07)
- [SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models](https://mlsanity.com/paper/2610.09600/) (2026-10-07)
- [The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models](https://mlsanity.com/paper/2610.07723/) (2026-10-06)
- [Reward Stealing Attack on Large Language Models](https://mlsanity.com/paper/2610.06670/) (2026-10-05)
- [Visual Grounding Safety in Vision-Language Models](https://mlsanity.com/paper/2610.05637/) (2026-10-05)
- [Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment](https://mlsanity.com/paper/2610.07023/) (2026-10-04)
- [Safe Context Switching for Agents in the Wild: Mitigating Subspace Interference via Orthogonal Adaptation](https://mlsanity.com/paper/2610.05219/) (2026-10-04)
- [EmoRSS: Mitigating Emotion-Induced Over-Refusal in Large Language Models](https://mlsanity.com/paper/2610.04998/) (2026-10-04)
- [Target-free Latent Safety Alignment](https://mlsanity.com/paper/2610.04467/) (2026-10-03)
- [DNAlign: Dynamic Null-Space Safe Alignment for LLMs](https://mlsanity.com/paper/2610.02844/) (2026-10-02)
- [TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety](https://mlsanity.com/paper/2610.01323/) (2026-10-01)
- [CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion](https://mlsanity.com/paper/2609.39902/) (2026-09-30)
- [ShieldCLIP: Selective Safety Alignment for Harmful Content Mitigation in Multimodal Foundation Models](https://mlsanity.com/paper/2609.39688/) (2026-09-30)
- [Faithful Dual-constrained Erasure for Robust LLM Safety Alignment](https://mlsanity.com/paper/2609.39279/) (2026-09-30)
- [ACTR: Aligning Thoughts and Responses for Multilingual Safety in Reasoning LLMs](https://mlsanity.com/paper/2609.37054/) (2026-09-30)
- [Refusal Localizes, the Damage Relocates: Safety Layers Under Few-Sample Fine-Tuning](https://mlsanity.com/paper/2610.00320/) (2026-09-29)
- [The Geometry of Harmfulness in Multi-Turn Attacks](https://mlsanity.com/paper/2609.38389/) (2026-09-29)
- [Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment](https://mlsanity.com/paper/2609.37837/) (2026-09-29)
- [Controlled Decoding Attacks on Black-Box LLMs](https://mlsanity.com/paper/2609.36956/) (2026-09-29)
- [Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning](https://mlsanity.com/paper/2609.36862/) (2026-09-29)
- [Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue](https://mlsanity.com/paper/2609.36849/) (2026-09-29)
- [Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models](https://mlsanity.com/paper/2609.36254/) (2026-09-28)
- [DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models](https://mlsanity.com/paper/2609.34896/) (2026-09-28)
- [When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety](https://mlsanity.com/paper/2609.34771/) (2026-09-28)
- [EOPSA: Efficient On-Policy Self-Distilled Safety Alignment](https://mlsanity.com/paper/2609.34519/) (2026-09-28)
- [AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents](https://mlsanity.com/paper/2609.33658/) (2026-09-27)
- [SafeMol: Dual-Modality Safety Alignment for Molecular Multimodal Models](https://mlsanity.com/paper/2609.33640/) (2026-09-27)
- [Understanding the Role of Prompt Template in Knowledge Distillation for Safety Alignment](https://mlsanity.com/paper/2609.30802/) (2026-09-25)