mlsanityData to 7 Oct 2026

SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation

arXiv 2501.01765 ↗First posted 3 Jan 2025cs.LGcited by 39

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

As advancements in large language models (LLMs) continue and the demand for personalized models increases, parameter-efficient fine-tuning (PEFT) methods (e.g., LoRA) will become essential due to their efficiency in reducing computation costs. However, recent studies have raised alarming concerns that LoRA fine-tuning could potentially compromise the safety alignment in LLMs, posing significant risks for the model owner. In this paper, we first investigate the underlying mechanism by analyzing the changes in safety alignment related features before and after fine-tuning. Then, we propose a fixed safety module calculated by safety data and a task-specific initialization for trainable parameters in low-rank adaptations, termed Safety-alignment preserved Low-Rank Adaptation (SaLoRA). Unlike previous LoRA methods and their variants, SaLoRA enables targeted modifications to LLMs without disrupting their original alignments. Our experiments show that SaLoRA outperforms various adapters-based approaches across various evaluation metrics in different fine-tuning tasks.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 47 of its 53 references that are in the map. First posted on arXiv 3 Jan 2025.

Builds on

47 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  2. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2023Mistral 7Bcited by 3,065
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
32 more
  1. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  5. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  6. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  7. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  9. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  10. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  11. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  12. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  13. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
  14. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  15. 2023VeRA: Vector-based Random Matrix Adaptationcited by 294
  16. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 261
  17. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  18. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  19. 2023Cones: Concept Neurons in Diffusion Models for Customized Generationcited by 150
  20. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  21. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  22. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  23. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  24. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 92
  25. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  26. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 78
  27. 2023Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterizationcited by 64
  28. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  29. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  30. 2024Dissecting Language Models: Machine Unlearning via Selective Pruningcited by 38
  31. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
  32. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Datacited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 19.

39 papers.

Sort:

By area:

  1. 2025Model Organisms for Emergent Misalignmentcited by 84
  2. 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistencycited by 25
  3. 2025Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safetycited by 22
  4. 2025Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbationcited by 20
  5. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 13
  6. 2025Unveiling the Basin-Like Loss Landscape in Large Language Modelscited by 13
  7. 2025JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoringcited by 12
  8. 2025CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuningcited by 11
  9. 2025Gradient Surgery for Safe LLM Fine-Tuningcited by 11
  10. 2025A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Spacecited by 9
  11. 2025Defending MoE LLMs against Harmful Fine-Tuning via Safety Routing Alignmentcited by 8
  12. 2025MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generationcited by 8
27 more
  1. 2025Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Studycited by 6
  2. 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6
  3. 2026Sparse Models, Sparse Safety: Unsafe Routes in Mixture-of-Experts LLMscited by 6
  4. 2026Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influencecited by 6
  5. 2025Curvature-Aware Safety Restoration In LLMs Fine-Tuningcited by 5
  6. 2025Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuitscited by 4
  7. 2026Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarkscited by 3
  8. 2025Document Retrieval Augmented Fine-Tuning (DRAFT) for safety-critical software assessmentscited by 2
  9. 2025Safety-Aligned Weights Are Not Enough: Refusal-Teacher-Guided Finetuning Enhances Safety and Downstream Performance under Harmful Finetuning Attackscited by 2
  10. 2026Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanismscited by 2
  11. 2025SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuningcited by 1
  12. 2026TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tamperingcited by 1
  13. 2026Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMscited by 1
  14. 2026Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directionscited by 1
  15. 2026Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgettingcited by 1
  16. 2026Safin-1: Safety from Within through Memory-Native State Evolutioncited by 1
  17. 2025Constrained Policy Optimization via Sampling-Based Weight-Space Projectioncited by 0
  18. 2026SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Modelscited by 0
  19. 2026You Snooze, You Lose: Automatic Safety Alignment Restoration through Neural Weight Translationcited by 0
  20. 2026The Attacker in the Mirror: Breaking Self-Consistency in Safety via Anchored Bipolicy Self-Playcited by 0
  21. 2026GradShield: Alignment Preserving Finetuningcited by 0
  22. 2026Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Modelscited by 0
  23. 2026TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignmentcited by 0
  24. 2026Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Studycited by 0
  25. 2026SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMscited by 0
  26. 2026Beyond Average Safety: Chance-Constrained LLM Fine-tuningcited by 0
  27. 2026SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Safe Pruning LoRA: Robust Distance-Guided Pruning for Safety Alignment in Adaptation of LLMs
  2. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
  3. 2026LSSF: Safety Alignment for Large Language Models through Low-Rank Safety Subspace Fusion
  4. 2024Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models
  5. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
  6. 2025Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
  7. 2025Safe at the Margins: A General Approach to Safety Alignment in Low-Resource English Languages -- A Singlish Case Study
  8. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching