mlsanityData to 7 Oct 2026

The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning

arXiv 2403.03218 ↗First posted 5 Mar 2024cs.LG cs.AI cs.CLcited by 488

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

The White House Executive Order on Artificial Intelligence highlights the risks of large language models (LLMs) empowering malicious actors in developing biological, cyber, and chemical weapons. To measure these risks of malicious use, government institutions and major AI labs are developing evaluations for hazardous capabilities in LLMs. However, current evaluations are private, preventing further research into mitigating risk. Furthermore, they focus on only a few, highly specific pathways for malicious use. To fill these gaps, we publicly release the Weapons of Mass Destruction Proxy (WMDP) benchmark, a dataset of 3,668 multiple-choice questions that serve as a proxy measurement of hazardous knowledge in biosecurity, cybersecurity, and chemical security. WMDP was developed by a consortium of academics and technical consultants, and was stringently filtered to eliminate sensitive information prior to public release. WMDP serves two roles: first, as an evaluation for hazardous knowledge in LLMs, and second, as a benchmark for unlearning methods to remove such hazardous knowledge. To guide progress on unlearning, we develop RMU, a state-of-the-art unlearning method based on controlling model representations. RMU reduces model performance on WMDP while maintaining general capabilities in areas such as biology and computer science, suggesting that unlearning may be a concrete path towards reducing malicious use from LLMs. We release our benchmark and code publicly at https://wmdp.ai

Key facts. Cited by 488 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 59 of its 89 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

59 of 89 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
38 more
  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2022Editing Models with Task Arithmeticcited by 1,076
  11. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  12. 2023Large Language Modelscited by 774
  13. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  14. 2020Aligning AI With Shared Human Valuescited by 706
  15. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  16. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  17. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  18. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 366
  19. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  20. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  21. 2021Unsolved Problems in ML Safetycited by 291
  22. 2023Large Language Model Unlearningcited by 288
  23. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  24. 2023Towards Unbounded Machine Unlearningcited by 261
  25. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  26. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  27. 2023In-Context Unlearning: Language Models as Few Shot Unlearnerscited by 214
  28. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  29. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampeningcited by 177
  30. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  31. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 171
  32. 2023Model evaluation for extreme riskscited by 167
  33. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
  34. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  35. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 143
  36. 2024Towards Safer Large Language Models through Machine Unlearningcited by 134
  37. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
  38. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 93; 2025: 182; 2026: 213.

488 papers.

Sort:

By area:

  1. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  2. 2025Humanity's Last Examcited by 467
  3. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  4. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  5. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearningcited by 160
  6. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  7. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  8. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  9. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  10. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 112
  11. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  12. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
38 more
  1. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  2. 2024CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Modelscited by 84
  3. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 81
  4. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  5. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 80
  6. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  7. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  8. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 71
  9. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  10. 2024Do Unlearning Methods Remove Information from Language Model Weights?cited by 66
  11. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Benchcited by 61
  12. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyondcited by 59
  13. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  14. 2024Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?cited by 56
  15. 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Studycited by 54
  16. 2024A Closer Look at Machine Unlearning for Large Language Modelscited by 54
  17. 2024On Large Language Model Continual Unlearningcited by 51
  18. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  19. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 50
  20. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  21. 2024MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Controlcited by 49
  22. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  23. 2024Sabotage Evaluations for Frontier Modelscited by 45
  24. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  25. 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 42
  26. 2024Soft Prompting for Unlearning in Large Language Modelscited by 42
  27. 2024Evaluating Copyright Takedown Methods for Language Modelscited by 42
  28. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMscited by 41
  29. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 39
  30. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  31. 2025LLM Unlearning via Neural Activation Redirectioncited by 38
  32. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 36
  33. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  34. 2025Safety Pretraining: Toward the Next Generation of Safe AIcited by 35
  35. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 34
  36. 2024Erasing Conceptual Knowledge from Language Modelscited by 34
  37. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localizationcited by 34
  38. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33

50 most cited of 488 All 488 (JSON)