mlsanityData to 7 Oct 2026

TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering

arXiv 2602.06911 ↗First posted 6 Feb 2026cs.CR cs.AIcited by 1

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

As increasingly capable open-weight large language models (LLMs) are deployed, improving their tamper resistance against unsafe modifications, whether accidental or intentional, becomes critical to minimize risks. However, there is no standard approach to evaluate tamper resistance. Varied datasets, metrics, and tampering configurations make it difficult to compare safety, utility, and robustness across different models and defenses. To address this, we introduce TamperBench, the first unified framework to systematically evaluate the tamper resistance of LLMs. TamperBench (i) curates a repository of state-of-the-art weight-space fine-tuning attacks, latent-space representation attacks, and alignment-stage defenses; (ii) enables realistic adversarial evaluation through systematic hyperparameter sweeps per attack-model pair; and (iii) provides both safety and utility evaluations. We use TamperBench to evaluate 21 open-weight LLMs, including defense-augmented variants, across nine tampering threats using standardized safety and capability metrics with hyperparameter sweeps per model-attack pair. The results provide insights including effects of post-training on tamper resistance, that jailbreak-tuning is typically the most severe attack, and that current alignment-stage defenses largely fail to withstand attack sweeps. Code is available at https://github.com/criticalml-uw/TamperBench.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 61 of its 75 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

61 of 75 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibility15 Jul
  2. 2024Tamper-Resistant Safeguards for Open-Weight LLMs1 Aug
  3. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs22 Jul
  4. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  5. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  7. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 54

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  10. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
38 more
  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  3. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  4. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  5. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 447
  6. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  7. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 261
  8. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 252
  9. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 171
  10. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 138
  11. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  12. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  13. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
  14. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  15. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  16. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 89
  17. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  18. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  19. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 81
  20. 2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 74
  21. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 71
  22. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  23. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  24. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  25. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 43
  26. 2024Obfuscated Activations Bypass LLM Latent-Space Defensescited by 40
  27. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptationcited by 39
  28. 2025Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMscited by 37
  29. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  30. 2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbationcited by 35
  31. 2024Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attackscited by 33
  32. 2024Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignmentcited by 31
  33. 2024Scaling Trends for Data Poisoning in LLMscited by 26
  34. 2025SubTrack++ : Gradient Subspace Tracking for Scalable LLM Trainingcited by 14
  35. 2025SDD: Self-Degraded Defense against Malicious Fine-tuningcited by 13
  36. 2024Revisiting the Robust Alignment of Circuit Breakerscited by 10
  37. 2025Fundamental Limitations in Pointwise Defences of LLM Finetuning APIscited by 10
  38. 2024Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuningcited by 8

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Modelscited by 0