mlsanityData to 14 Oct 2026

Continual Safety Alignment via Gradient-Based Sample Selection

arXiv 2604.17215 ↗First posted 19 Apr 2026cs.LGcited by 4

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Large language models require continuous adaptation to new tasks while preserving safety alignment. However, fine-tuning on even benign data often compromises safety behaviors, including refusal of harmful requests, truthfulness, and commonsense reasoning. We investigate which training samples cause alignment drift through a data-centric lens. Our empirical analysis shows samples contribute unequally: high-gradient samples cause greater safety degradation and drive models toward pretrained distributions, while moderate-gradient samples enable task learning with minimal alignment loss. We propose gradient-based sample selection that filters high-gradient samples during fine-tuning. Across multiple model families on continual domain tasks, our method substantially improves alignment preservation while maintaining competitive task performance, without requiring curated safe data or architectural modifications. Our method is robust across selection ratios, task orderings, and diverse attack benchmarks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 30 of its 43 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

30 of 43 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Curvature-Aware Safety Restoration In LLMs Fine-Tuning22 Nov
  2. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets5 Jun
  3. 2025Shape it Up! Restoring LLM Safety during Finetuning22 May
  4. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning18 Aug
  5. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack28 May
  6. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  8. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models4 Oct

Other references · 22

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  10. 2018Experience Replay for Continual Learningcited by 1,095
  11. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  12. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
10 more
  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  2. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 156
  3. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
  4. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 96
  5. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  6. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 53
  7. 2025Spurious Forgetting in Continual Learning of Language Modelscited by 51
  8. 2025Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safetycited by 22
  9. 2025Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasetscited by 18
  10. 2025Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026A Gravitational Interpretation of Safety Reversion under Fine-Tuningcited by 0
  2. 2026Multilingual Safety Signals Are Multi-Layered: Filtering Safety-Degrading Data for Safer LLMscited by 0
  3. 2026Why Jailbreaks Succeed in Diffusion Language Models: An Energy Landscape Analysiscited by 0
  4. 2026MASCRDM: Multi-Agent System for Compliance Risk Detection and Mitigation in Training Process of Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection
  2. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
  3. 2026SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
  4. 2025Unforgotten Safety: Preserving Safety Alignment of Large Language Models with Continual Learning
  5. 2025Gradient Surgery for Safe LLM Fine-Tuning
  6. 2026Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment
  7. 2026Two to Tango: Coupled Task-Reference Selection for Safe LLM Fine-tuning
  8. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance