mlsanityData to 14 Oct 2026

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models

arXiv 2503.20807 ↗First posted 24 Mar 2025stat.ML cs.AI cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Fine-tuning Large Language Models (LLMs) on some task-specific datasets has been a primary use of LLMs. However, it has been empirically observed that this approach to enhancing capability inevitably compromises safety, a phenomenon also known as the safety-capability trade-off in LLM fine-tuning. This paper presents a theoretical framework for understanding the interplay between safety and capability in two primary safety-aware LLM fine-tuning strategies, providing new insights into the effects of data similarity, context overlap, and alignment loss landscape. Our theoretical results characterize the fundamental limits of the safety-capability trade-off in LLM fine-tuning, which are also validated by numerical experiments.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 12 of its 24 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

12 of 24 references.

References · 12

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  6. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  7. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  8. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  9. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
  10. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 90
  11. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  12. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Modelscited by 46

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 8.

17 papers.

Sort:

By area:

  1. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 20
  2. 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systemscited by 14
  3. 2025Unveiling the Basin-Like Loss Landscape in Large Language Modelscited by 13
  4. 2025Shape it Up! Restoring LLM Safety during Finetuningcited by 11
  5. 2025When Thinking Backfires: Mechanistic Insights Into Reasoning-Induced Misalignmentcited by 6
  6. 2026Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influencecited by 6
  7. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learningcited by 4
  8. 2025SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimizationcited by 3
  9. 2026Mitigating Safety Tax via Distribution-Grounded Refinement in Large Reasoning Modelscited by 3
  10. 2025ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Conceptscited by 2
  11. 2025Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Modelscited by 2
  12. 2026GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuningcited by 2
5 more
  1. 2026Alignment Dynamics in LLM Fine-Tuningcited by 2
  2. 2026Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuningcited by 0
  3. 2026From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuningcited by 0
  4. 2026The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systemscited by 0
  5. 2026Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Trainingcited by 0