mlsanityData to 7 Oct 2026

TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment

arXiv 2607.16242 ↗First posted 26 Jun 2026cs.LG cs.AI cs.CRcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Fine-Tuning-as-a-Service (FTaaS) platforms let users perform supervised fine-tuning (SFT) on customized data, but this pipeline can erode model safety alignment. To recover safety without re-running full alignment, existing realignment methods focus on calibrating the integration of safety patches into fine-tuned models. These methods exhibit a persistent safety-utility trade-off: weak repair leaves harmful behavior intact, while stronger repair increasingly damages the benign task. This paper shifts the focus from online calibration to offline patch learning and aims to learn a safety patch that restores safety while preserving task-specific capabilities. To this end, we propose TRACE, which simulates harmful SFT trajectories to produce progressively corrupted model states, and optimizes a safety patch simultaneously across these states. TRACE trains the safety patch during the offline stage, and reuses it across all user checkpoints without per-user calibration. We evaluate two representative models using three harmful SFT datasets, together with three utility benchmarks. Across six benchmarks and two models, TRACE consistently dominates the safety-utility frontier. TRACE improves the safety rate by up to 77 percentage points over the best baselines, while maintaining comparable utility to the undefended fine-tuned model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 38 of its 57 references that are in the map. First posted on arXiv 26 Jun 2026.

Builds on

38 of 57 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance5 Jan
  2. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference20 Jun
  3. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic19 Feb
  4. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack2 Feb
  5. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  7. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models4 Oct
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 29

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  10. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
17 more
  1. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  2. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  3. 2022Editing Models with Task Arithmeticcited by 1,076
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2022Toy Models of Superpositioncited by 928
  6. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  7. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  8. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
  9. 2024A StrongREJECT for Empty Jailbreakscited by 352
  10. 2026Qwen3.5-Omni Technical Reportcited by 146
  11. 2025Qwen3Guard Technical Reportcited by 125
  12. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  13. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  14. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  15. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptationcited by 39
  16. 2025Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasetscited by 17
  17. 2026Understanding and Preserving Safety in Fine-Tuned LLMscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
  2. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
  3. 2025Shape it Up! Restoring LLM Safety during Finetuning
  4. 2026Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
  5. 2025Gradient Surgery for Safe LLM Fine-Tuning
  6. 2025Curvature-Aware Safety Restoration In LLMs Fine-Tuning
  7. 2026The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training
  8. 2026Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment