mlsanityData to 14 Oct 2026

SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning

arXiv 2506.00676 ↗First posted 31 May 2025cs.LG cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

As large language models (LLMs) become ubiquitous, parameter-efficient fine-tuning methods and safety-first defenses have proliferated rapidly. However, the number of approaches and their recent increase have resulted in diverse evaluations-varied datasets, metrics, and inconsistent threat settings-making it difficult to fairly compare safety, utility, and robustness across methods. To address this, we introduce SafeTuneBed, a benchmark and toolkit unifying fine-tuning and defense evaluation. SafeTuneBed (i) curates a diverse repository of multiple fine-tuning datasets spanning sentiment analysis, question-answering, multi-step reasoning, and open-ended instruction tasks, and allows for the generation of harmful-variant splits; (ii) enables integration of state-of-the-art defenses, including alignment-stage immunization, in-training safeguards, and post-tuning repair; and (iii) provides evaluators for safety (attack success rate, refusal consistency) and utility. Built on Python-first, dataclass-driven configs and plugins, SafeTuneBed requires minimal additional code to specify any fine-tuning regime, defense method, and metric suite, while ensuring end-to-end reproducibility. We showcase its value by benchmarking representative defenses across varied poisoning scenarios and tasks. By standardizing data, code, and metrics, SafeTuneBed is the first focused toolkit of its kind to accelerate rigorous and comparable research in safe LLM fine-tuning. Code is available at: https://github.com/criticalml-uw/SafeTuneBed

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 42 of its 62 references that are in the map. First posted on arXiv 31 May 2025.

Builds on

42 of 62 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation13 Oct
  2. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey26 Sep
  3. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Security30 Aug
  4. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning18 Aug
  5. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  6. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack28 May
  7. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curation24 May
  8. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  9. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack2 Feb
  10. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  12. 2023Are aligned neural networks adversarially aligned?26 Jun
1 more
  1. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  10. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  11. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  12. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
17 more
  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  2. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 463
  3. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  4. 2017Certified Defenses for Data Poisoning Attackscited by 303
  5. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 275
  6. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 269
  7. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 138
  8. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
  9. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 96
  10. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 96
  11. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  12. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 88
  13. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  14. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptationcited by 42
  15. 2024Immunization against harmful fine-tuning attackscited by 35
  16. 2024BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Modelscited by 27
  17. 2024Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Can Large Language Models Make Everyone Happy?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM-Safety Evaluations Lack Robustness
  2. 2025TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesis
  3. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
  4. 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
  5. 2024Locking Down the Finetuned LLMs Safety
  6. 2026RAS: Measuring LLM Safety Through Refusal Alignment
  7. 2026Understanding and Preserving Safety in Fine-Tuned LLMs
  8. 2026SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs