mlsanityData to 7 Oct 2026

Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models

arXiv 2211.14946 ↗First posted 27 Nov 2022cs.LGcited by 56

Not on the map. Why?

Cited by 29 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A growing ecosystem of large, open-source foundation models has reduced the labeled data and technical expertise necessary to apply machine learning to many new problems. Yet foundation models pose a clear dual-use risk, indiscriminately reducing the costs of building both harmful and beneficial machine learning systems. Policy tools such as restricted model access and export controls are the primary methods currently used to mitigate such dual-use risks. In this work, we review potential safe-release strategies and argue that both policymakers and AI researchers would benefit from fundamentally new technologies enabling more precise control over the downstream usage of open-source foundation models. We propose one such approach: the task blocking paradigm, in which foundation models are trained with an additional mechanism to impede adaptation to harmful tasks without sacrificing performance on desirable tasks. We call the resulting models self-destructing models, inspired by mechanisms that prevent adversaries from using tools for harmful purposes. We present an algorithm for training self-destructing models leveraging techniques from meta-learning and adversarial learning, which we call meta-learned adversarial censoring (MLAC). In a small-scale experiment, we show MLAC can largely prevent a BERT-style model from being re-purposed to perform gender identification without harming the model's ability to perform profession classification.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 36 of its 70 references that are in the map. First posted on arXiv 27 Nov 2022.

Builds on

36 of 70 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
24 more
  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  3. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  4. 2021Editing Factual Knowledge in Language Modelscited by 548
  5. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  6. 2021Fast Model Editing at Scalecited by 491
  7. 2019Differentiable Convex Optimization Layerscited by 458
  8. 2022Memory-Based Model Editing at Scalecited by 443
  9. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
  10. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  11. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  12. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  13. 2015Censoring Representations with an Adversarycited by 191
  14. 2020Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claimscited by 163
  15. 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 159
  16. 2020Editable Neural Networkscited by 154
  17. 2019Meta-Learning with Warped Gradient Descentcited by 98
  18. 2022Linear Adversarial Concept Erasurecited by 98
  19. 2019Generalized Inner Loop Meta-Learningcited by 92
  20. 2019Meta-Curvaturecited by 70
  21. 2022Kernelized Concept Erasurecited by 47
  22. 2019Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learningcited by 12
  23. 2022How Would The Viewer Feel? Estimating Wellbeing From Video Scenarioscited by 9
  24. 2019The tension between openness and prudence in AI researchcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 24; 2025: 15; 2026: 7.

56 papers.

Sort:

By area:

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  3. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  4. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  5. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  6. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  7. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  8. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  9. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  10. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 105
  11. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 93
  12. 2024Guardrail Baselines for Unlearning in LLMscited by 90
38 more
  1. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  2. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  3. 2024Do Unlearning Methods Remove Information from Language Model Weights?cited by 66
  4. 2023PIGEON: Predicting Image Geolocationscited by 55
  5. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  6. 2024UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AIcited by 46
  7. 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 42
  8. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 39
  9. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localizationcited by 34
  10. 2024Immunization against harmful fine-tuning attackscited by 33
  11. 2024Backtracking Improves Generation Safetycited by 31
  12. 2024No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attackscited by 28
  13. 2024Gradient Routing: Masking Gradients to Localize Computation in Neural Networkscited by 28
  14. 2024Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?cited by 24
  15. 2025Estimating Worst-Case Frontier Risks of Open-Weight LLMscited by 17
  16. 2025SDD: Self-Degraded Defense against Malicious Fine-tuningcited by 13
  17. 2025Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluationcited by 11
  18. 2025CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuningcited by 11
  19. 2025Self-Destructive Language Modelcited by 10
  20. 2023Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Modelscited by 7
  21. 2025Existing Large Language Model Unlearning Evaluations Are Inconclusivecited by 7
  22. 2024Locking Machine Learning Models into Hardwarecited by 6
  23. 2024Applying Refusal-Vector Ablation to Llama 3.1 70B Agentscited by 5
  24. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  25. 2024Evaluating Defences against Unsafe Feedback in RLHFcited by 4
  26. 2025Robust LLM Unlearning with MUDMAN: Meta-Unlearning with Disruption Masking And Normalizationcited by 4
  27. 2023Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processingcited by 3
  28. 2023Censoring chemical data to mitigate dual use riskcited by 3
  29. 2025Collapse of Irrelevant Representations (CIR) Ensures Robust and Non-Disruptive LLM Unlearningcited by 3
  30. 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 3
  31. 2025NLP Security and Ethics, in the Wildcited by 2
  32. 2025A Comparative Analysis of Ethical and Safety Gaps in LLMs using Relative Danger Coefficientcited by 2
  33. 2023The Promise and Peril of Artificial Intelligence -- Violet Teaming Offers a Balanced Path Forwardcited by 1
  34. 2025Beyond Release: Access Considerations for Generative AI Systemscited by 1
  35. 2025Patronus: Safeguarding Text-to-Image Models against White-Box Adversariescited by 1
  36. 2026TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tamperingcited by 1
  37. 2026A Few Steps Further: Why Defenses Against Malicious Finetuning Erode Under Continued Trainingcited by 1
  38. 2026Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attackscited by 1

50 most cited of 56 All 56 (JSON)