# Protecting Society from AI Misuse: When are Restrictions on Capabilities Warranted?

arXiv: https://arxiv.org/abs/2303.09377 · HTML: https://mlsanity.com/paper/2303.09377/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 138 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 15 of 15)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Predictability and Surprise in Large Generative Models](https://mlsanity.com/paper/2202.07785/) (2022)
- [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://mlsanity.com/paper/1802.07228/) (2018)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations](https://mlsanity.com/paper/2301.04246/) (2023)
- [Responsible Disclosure of Generative Models Using Scalable Fingerprinting](https://mlsanity.com/paper/2012.08726/) (2020)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [Structured access: an emerging paradigm for safe AI deployment](https://mlsanity.com/paper/2201.05159/) (2022)
- [Institutionalising Ethics in AI through Broader Impact Requirements](https://mlsanity.com/paper/2106.11039/) (2021)
- [On the Horizon: Interactive and Compositional Deepfakes](https://mlsanity.com/paper/2209.01714/) (2022)
- [The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse?](https://mlsanity.com/paper/2001.00463/) (2019)
- [It's Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Process](https://mlsanity.com/paper/2112.09544/) (2021)
- [Generative Models for Spear Phishing Posts on Social Media](https://mlsanity.com/paper/1802.05196/) (2018)

## Built on by (most cited first, 16 of 16)

- [Frontier AI Regulation: Managing Emerging Risks to Public Safety](https://mlsanity.com/paper/2307.03718/) (2023)
- [Spear Phishing With Large Language Models](https://mlsanity.com/paper/2305.06972/) (2023)
- [Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives](https://mlsanity.com/paper/2311.09227/) (2023)
- [Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data](https://mlsanity.com/paper/2406.13843/) (2024)
- [Societal Adaptation to Advanced AI](https://mlsanity.com/paper/2405.10295/) (2024)
- [Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models](https://mlsanity.com/paper/2312.14751/) (2023)
- [Just as Humans Need Vaccines, So Do Models: Model Immunization to Combat Falsehoods](https://mlsanity.com/paper/2505.17870/) (2025)
- [Gen-AI for User Safety: A Survey](https://mlsanity.com/paper/2411.06606/) (2024)
- [Contextual Confidence and Generative AI](https://mlsanity.com/paper/2311.01193/) (2023)
- [Fast Sampling Through The Reuse Of Attention Maps In Diffusion Models](https://mlsanity.com/paper/2401.01008/) (2023)
- [Position: It's Time to Act on the Risk of Efficient Personalized Text Generation](https://mlsanity.com/paper/2502.06560/) (2025)
- [Beyond Release: Access Considerations for Generative AI Systems](https://mlsanity.com/paper/2502.16701/) (2025)
- [From Turing to Tomorrow: The UK's Approach to AI Regulation](https://mlsanity.com/paper/2507.03050/) (2025)
- [A Peek Behind the Curtain: Using Step-Around Prompt Engineering to Identify Bias and Misinformation in GenAI Models](https://mlsanity.com/paper/2503.15205/) (2025)
- [Know Your Scientist: KYC as Biosecurity Infrastructure](https://mlsanity.com/paper/2602.06172/) (2026)
- [Rethinking AI Literacy Education in Higher Education: Bridging Risk Perception and Responsible Adoption](https://mlsanity.com/paper/2603.29935/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI) systems will increasingly be used to cause harm as they grow more capable. In fact, AI systems are already starting to be used to automate fraudulent activities, violate human rights, create harmful fake images, and identify dangerous toxins. To prevent some misuses of AI, we argue that targeted interventions on certain capabilities will be warranted. These restrictions may include controlling who can access certain types of AI models, what they can be used for, whether outputs are filtered or can be traced back to their user, and the resources needed to develop them. We also contend that some restrictions on non-AI capabilities needed to cause harm will be required. Though capability restrictions risk reducing use more than misuse (facing an unfavorable Misuse-Use Tradeoff), we argue that interventions on capabilities are warranted when other interventions are insufficient, the potential harm from misuse is high, and there are targeted ways to intervene on capabilities. We provide a taxonomy of interventions that can reduce AI misuse, focusing on the specific steps required for a misuse to cause harm (the Misuse Chain), and a framework to determine if an intervention is warranted. We apply this reasoning to three examples: predicting novel toxins, creating harmful images, and automating spear phishing campaigns.