mlsanityData to 7 Oct 2026

Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching

arXiv 2405.13820 ↗First posted 22 May 2024cs.CLcited by 16

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety alignment of large language models (LLMs) has been gaining increasing attention. However, current safety-aligned LLMs suffer from the fragile and imbalanced safety mechanisms, which can still be induced to generate unsafe responses, exhibit over-safety by rejecting safe user inputs, and fail to preserve general utility after safety alignment. To this end, we propose a novel post safety alignment (PSA) method to address these inherent and emerging safety challenges, including safety enhancement, over-safety mitigation, and utility preservation. In specific, we introduce textscSafePatching, a novel framework for comprehensive PSA, where two distinct safety patches are developed on the harmful data to enhance safety and mitigate over-safety concerns, and then seamlessly integrated into the target LLM backbone without compromising its utility. Extensive experiments on four representative aligned LLMs, including LLaMA-2/3, Gemma and Mistral, show that textscSafePatching achieves a more comprehensive PSA than baseline methods, further optimizing the balance between being helpful and harmless in current aligned LLMs. Also, textscSafePatching demonstrates its superiority in continual PSA scenarios.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 55 of its 84 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

55 of 84 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic19 Feb
  2. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  3. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  6. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2020Longformer: The Long-Document Transformercited by 3,362
37 more
  1. 2023Mistral 7Bcited by 3,065
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  8. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  9. 2022Editing Models with Task Arithmeticcited by 1,076
  10. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  11. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  12. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  13. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  14. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  15. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  16. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  17. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  18. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  19. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  20. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  21. 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 384
  22. 2023Large Language Model Unlearningcited by 288
  23. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  24. 2023Pretraining Language Models with Human Preferencescited by 245
  25. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  26. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  27. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  28. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 211
  29. 2022Patching open-vocabulary models by interpolating weightscited by 187
  30. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  31. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  32. 2024Towards Safer Large Language Models through Machine Unlearningcited by 134
  33. 2024Navigating the OverKill in Large Language Modelscited by 40
  34. 2024ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decodingcited by 32
  35. 2024HFT: Half Fine-Tuning for Large Language Modelscited by 18
  36. 2024From Representational Harms to Quality-of-Service Harms: A Case Study on Llama 2 Safety Safeguardscited by 17
  37. 2024Mitigating Exaggerated Safety in Large Language Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 5; 2026: 4.

16 papers.

Sort:

By area:

  1. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activationscited by 32
  2. 2024To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Modelscited by 25
  3. 2024Dissecting Fine-Tuning Unlearning in Large Language Modelscited by 25
  4. 2024Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixturecited by 22
  5. 2024Locking Down the Finetuned LLMs Safetycited by 22
  6. 2025RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguardscited by 14
  7. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  8. 2025Reviving Your MNEME: Predicting The Side Effects of LLM Unlearning and Fine-Tuning via Sparse Model Diffingcited by 8
  9. 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignmentcited by 6
  10. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  11. 2026When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agentscited by 5
  12. 2024The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Mergingcited by 3
4 more
  1. 2025Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmeticcited by 1
  2. 2025The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Modelscited by 1
  3. 2026Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correctioncited by 1
  4. 2026Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
  2. 2024Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models
  3. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
  4. 2026LSSF: Safety Alignment for Large Language Models through Low-Rank Safety Subspace Fusion
  5. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
  6. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements
  7. 2025Safety Alignment Depth in Large Language Models: A Markov Chain Perspective
  8. 2026Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection