# The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

arXiv: https://arxiv.org/abs/1802.07228 · HTML: https://mlsanity.com/paper/1802.07228/

## Key facts

- Cited by 165 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2018.

## Built on by (most cited first, 30 of 165)

- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Are aligned neural networks adversarially aligned?](https://mlsanity.com/paper/2306.15447/) (2023)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Model evaluation for extreme risks](https://mlsanity.com/paper/2305.15324/) (2023)
- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies](https://mlsanity.com/paper/2407.19354/) (2024)
- [Regulating ChatGPT and other Large Generative AI Models](https://mlsanity.com/paper/2302.02337/) (2023)
- [Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems](https://mlsanity.com/paper/2405.06624/) (2024)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends](https://mlsanity.com/paper/2407.07403/) (2024)
- [Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities](https://mlsanity.com/paper/2308.12833/) (2023)
- [Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis](https://mlsanity.com/paper/2208.13753/) (2022)
- [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025)
- [On the Societal Impact of Open Foundation Models](https://mlsanity.com/paper/2403.07918/) (2024)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [Introducing v0.5 of the AI Safety Benchmark from MLCommons](https://mlsanity.com/paper/2404.12241/) (2024)
- [X-Risk Analysis for AI Research](https://mlsanity.com/paper/2206.05862/) (2022)
- [Improved Transformer for High-Resolution GANs](https://mlsanity.com/paper/2106.07631/) (2021)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models](https://mlsanity.com/paper/2211.14946/) (2022)
- [SparCL: Sparse Continual Learning on the Edge](https://mlsanity.com/paper/2209.09476/) (2022)
- [Agentic Web: Weaving the Next Web with AI Agents](https://mlsanity.com/paper/2507.21206/) (2025)
- [Synthetic Disinformation Attacks on Automated Fact Verification Systems](https://mlsanity.com/paper/2202.09381/) (2022)
- [The Threat of Offensive AI to Organizations](https://mlsanity.com/paper/2106.15764/) (2021)
- [The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail](https://mlsanity.com/paper/2110.08300/) (2021)
- [Anatomy of an AI-powered malicious social botnet](https://mlsanity.com/paper/2307.16336/) (2023)

## Abstract (arXiv)

This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the ways in which AI may influence the threat landscape in the digital, physical, and political domains, we make four high-level recommendations for AI researchers and other stakeholders. We also suggest several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute. Finally, we discuss, but do not conclusively resolve, the long-term equilibrium of attackers and defenders.