Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
Not on the map. Why?
Abstract (arXiv)
Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal ablation and brevity amplification across five instruction-tuned models (3B-14B) under non-adversarial SFT and RLHF. We find that steering degrades more under stronger optimisation pressure against the steered behaviour, which depends on both the training data and the training algorithm. Refusal ablation loses 69% of its effect on average under our SFT setup, but only 2% under our RLHF setup. The weight-edit, by contrast, remains almost unchanged (on average only 0.4% of it is restored), and the fine-tuning update along the steering direction is close to orthogonal to the pre-edit weight pattern (mean cosine similarity 0.071). Fine-tuning that restores the behaviour does so without reversing the edit. As the steered behaviour is not fully durable, embedded steering should be re-validated after downstream training.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
24 of 30 references.
References · 24
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2023LIMA: Less Is More for Alignmentcited by 1,080
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
12 more
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
- 2024Programming Refusal with Conditional Activation Steeringcited by 191
- 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
- 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
- 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
- 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
- 2025An Embarrassingly Simple Defense Against LLM Abliteration Attackscited by 12
- 2025Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibilitycited by 6
- 2025Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injectioncited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026On the Steering Dimensionality of Refusal in Language Models
- 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
- 2026Steer2Edit: From Activation Steering to Component-Level Editing
- 2025The Rogue Scalpel: Activation Steering Compromises LLM Safety
- 2026FaithSteer-BENCH: A Deployment-Aligned Stress-Testing Benchmark for Inference-Time Steering
- 2026Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models
- 2026A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models
- 2025Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Models