mlsanityData to 7 Oct 2026

Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints

arXiv 2604.12384 ↗First posted 14 Apr 2026cs.AIcited by 2

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety alignment in Large Language Models (LLMs) remains highly fragile during fine-tuning, where even benign adaptation can degrade pre-trained refusal behaviors and enable harmful responses. Existing defenses typically constrain either weights or activations in isolation, without considering their coupled effects on safety. In this paper, we first theoretically demonstrate that constraining either weights or activations alone is insufficient for safety preservation. To robustly preserve safety alignment, we propose Coupled Weight and Activation Constraints (CWAC), a novel approach that simultaneously enforces a precomputed safety subspace on weight updates and applies targeted regularization to safety-critical features identified by sparse autoencoders. Extensive experiments across four widely used LLMs and diverse downstream tasks show that CWAC consistently achieves the lowest harmful scores with minimal impact on fine-tuning accuracy, substantially outperforming strong baselines even under high harmful data ratios.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 29 of its 41 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

29 of 41 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging21 Mar
  2. 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning17 Dec
  3. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey26 Sep
  4. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Security30 Aug
  5. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering21 Aug
  6. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning18 Aug
  7. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack28 May
  8. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  11. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 18

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  8. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  9. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  10. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  11. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 92
  12. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
6 more
  1. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 45
  2. 2025Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbationcited by 20
  3. 2025LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuningcited by 12
  4. 2025Defending MoE LLMs against Harmful Fine-Tuning via Safety Routing Alignmentcited by 8
  5. 2024SCAR: Sparse Conditioned Autoencoders for Concept Detection and Steering in LLMscited by 6
  6. 2025SC-LoRA: Balancing Efficient Fine-tuning and Knowledge Preservation via Subspace-Constrained LoRAcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMscited by 0
  2. 2026Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
  2. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
  3. 2026Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
  4. 2026Multilingual Safety Alignment Via Sparse Weight Editing
  5. 2026LSSF: Safety Alignment for Large Language Models through Low-Rank Safety Subspace Fusion
  6. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
  7. 2026SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
  8. 2024Locking Down the Finetuned LLMs Safety