mlsanityData to 7 Oct 2026

Badllama 3: removing safety finetuning from Llama 3 in minutes

arXiv 2407.01376 ↗First posted 1 Jul 2024cs.LG cs.AI cs.CLcited by 10

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

We show that extensive LLM safety fine-tuning is easily subverted when an attacker has access to model weights. We evaluate three state-of-the-art fine-tuning methods-QLoRA, ReFT, and Ortho-and show how algorithmic advances enable constant jailbreaking performance with cuts in FLOPs and optimisation power. We strip safety fine-tuning from Llama 3 8B in one minute and Llama 3 70B in 30 minutes on a single GPU, and sketch ways to reduce this further.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 22 of its 33 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

22 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct

Other references · 20

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2023LIMA: Less Is More for Alignmentcited by 1,080
  12. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
8 more
  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  4. 2023Steering Language Models With Activation Engineeringcited by 824
  5. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
  6. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  7. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  8. 2024CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Modelscited by 84

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 3.

10 papers.

  1. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  2. 2025Demonstrating specification gaming in reasoning modelscited by 48
  3. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  4. 2025AI Companies Should Report Pre- and Post-Mitigation Safety Evaluationscited by 3
  5. 2025Mitigating Cyber Risk in the Age of Open-Weight LLMs: Policy Gaps and Technical Realitiescited by 3
  6. 2026Segment-Level Coherence for Robust Harmful Intent Probing in LLMscited by 3
  7. 2024Countering Autonomous Cyber Threatscited by 1
  8. 2025Winning at All Cost: A Small Environment for Eliciting Specification Gaming Behaviors in Large Language Modelscited by 1
  9. 2026Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuningcited by 1
  10. 2026Code World Model Preparedness Reportcited by 0