mlsanityData to 14 Oct 2026

When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning

arXiv 2609.01455 ↗First posted 1 Sep 2026cs.CR cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Benign fine-tuning severely weakens the safety alignment of large language models (LLMs), so we study why refusal behavior is so fragile. While prior work often attributes this failure to gradient conflict, we propose a fundamentally different Fisher-geometric explanation: safety Fisher is low-rank, and alignment makes the safety geometry flatter while preserving an output-routing pathway. After 100 benign fine-tuning examples, this pathway is selectively re-sharpened in output-side MLP modules, explaining the asymmetric fragility: safety can collapse to high attack success rates, while general utility degrades mildly. The routing view also explains why few safety examples can restore refusal behavior, indicating that internal safety-relevant representations are preserved. Finally, we show that LoRA and ASAM mitigate early collapse by suppressing output-side sharpness, but their protection weakens at larger fine-tuning scales. Overall, safety failure is best understood as a disruption of a low-rank output-routing mechanism

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 24 of its 39 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

24 of 39 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety17 Feb
  2. 2025Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMs4 Jul
  3. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets5 Jun
  4. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Security30 Aug
  5. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  6. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  8. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models4 Oct
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  9. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  10. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 274
  11. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 261
  12. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 118
3 more
  1. 2023The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Riskscited by 36
  2. 2025Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safetycited by 22
  3. 2025Rethinking Safety in LLM Fine-tuning: An Optimization Perspectivecited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs
  2. 2026From Refusal Geometry to Safety Geometry: Harmfulness--Refusal Coupling under Dynamic Adversarial Fine-Tuning
  3. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
  4. 2026Understanding and Preserving Safety in Fine-Tuned LLMs
  5. 2026SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models
  6. 2026Localizing Safety Alignment: MLP Layers and Mid-Network Blocks Encode Refusal Behavior in Large Language Models
  7. 2026RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs
  8. 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness