mlsanityData to 7 Oct 2026

Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignment

arXiv 2506.00166 ↗First posted 30 May 2025cs.LG cs.AI cs.CLcited by 6

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Existing paradigms for ensuring AI safety, such as guardrail models and alignment training, often compromise either inference efficiency or development flexibility. We introduce Disentangled Safety Adapters (DSA), a novel framework addressing these challenges by decoupling safety-specific computations from a task-optimized base model. DSA utilizes lightweight adapters that leverage the base model's internal representations, enabling diverse and flexible safety functionalities with minimal impact on inference cost. Empirically, DSA-based safety guardrails substantially outperform comparably sized standalone models across hate speech classification, detecting unsafe model inputs and responses, and hallucination detection with relative improvements of up to 53% in AUC. Furthermore, DSA-based safety alignment allows dynamic, inference-time adjustment of alignment strength and a fine-grained trade-off between instruction following performance and model safety. Importantly, combining the DSA safety guardrail with DSA safety alignment facilitates context-dependent alignment strength, boosting safety on StrongREJECT by 93% while maintaining 98% performance on MTBench - a total reduction in alignment tax of 8 percentage points compared to standard safety alignment fine-tuning. Overall, DSA presents a promising path towards more modular, efficient, and adaptable AI safety and alignment.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 51 of its 57 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

51 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Towards Inference-time Category-wise Safety Steering for Large Language Models2 Oct
  2. 2024Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing11 Jul
  3. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity3 Jan
  4. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 47

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2024Qwen2.5 Technical Reportcited by 4,713
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  12. 2016Understanding intermediate layers using linear classifier probescited by 1,279
35 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  3. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  4. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  7. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  8. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  9. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  10. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  11. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  12. 2024A StrongREJECT for Empty Jailbreakscited by 352
  13. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  14. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  15. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  16. 2022LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learningcited by 228
  17. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  18. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  19. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  20. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  21. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
  22. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 122
  23. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  24. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  25. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  26. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  27. 2024Decoding-time Realignment of Language Modelscited by 67
  28. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  29. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  30. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 44
  31. 2023Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbonecited by 26
  32. 2024Granite Guardiancited by 20
  33. 2024Rapid Response: Mitigating LLM Jailbreaks with a Few Examplescited by 15
  34. 2024Lightweight Safety Classification Using Pruned Language Modelscited by 5
  35. 2024Logistic Regression makes small LLMs strong and explainable "tens-of-shot" classifierscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detectioncited by 6
  2. 2026SentGuard: Sentence-Level Streaming Guardrails for Large Language Modelscited by 2
  3. 2025Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Valuescited by 1
  4. 2026ROM: Real-time Overthinking Mitigation via Streaming Detection and Interventioncited by 1
  5. 2026Legislating World-Model-Based Planning with Legal Reasoningcited by 0
  6. 2026Constitutional adapters: Inference-time interventions for misalignment and misusecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignment
  2. 2026Safety Context Injection: Inference-Time Safety Alignment via Static Filtering and Agentic Analysis
  3. 2025Vulnerability Mitigation for Safety-Aligned Language Models via Debiasing
  4. 2025Noise Injection Systemically Degrades Large Language Model Safety Guardrails
  5. 2025SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
  6. 2025A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space
  7. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Models
  8. 2025Safety Alignment Can Be Not Superficial With Explicit Safety Signals