# A Survey on Offensive AI Within Cybersecurity

arXiv: https://arxiv.org/abs/2410.03566 · HTML: https://mlsanity.com/paper/2410.03566/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 97 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 20 of 20)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Parseval Networks: Improving Robustness to Adversarial Examples](https://mlsanity.com/paper/1704.08847/) (2017)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [Adversarial Attacks and Defences: A Survey](https://mlsanity.com/paper/1810.00069/) (2018)
- [GitHub Copilot AI pair programmer: Asset or Liability?](https://mlsanity.com/paper/2206.15331/) (2022)
- [Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities](https://mlsanity.com/paper/2308.12833/) (2023)
- [Stealing Neural Networks via Timing Side Channels](https://mlsanity.com/paper/1812.11720/) (2018)
- [On the Impossible Safety of Large AI Models](https://mlsanity.com/paper/2209.15259/) (2022)
- [Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing](https://mlsanity.com/paper/2004.14942/) (2020)
- [On managing vulnerabilities in AI/ML systems](https://mlsanity.com/paper/2101.10865/) (2021)
- [The Security of Deep Learning Defences for Medical Imaging](https://mlsanity.com/paper/2201.08661/) (2022)
- [Online Abuse toward Candidates during the UK General Election 2019: Working Paper](https://mlsanity.com/paper/2001.08686/) (2020)
- [Data Breaches in Healthcare Security Systems](https://mlsanity.com/paper/2111.00582/) (2021)
- [Identifying and examining machine learning biases on Adult dataset](https://mlsanity.com/paper/2310.09373/) (2023)
- [Intell-dragonfly: A Cybersecurity Attack Surface Generation Engine Based On Artificial Intelligence-generated Content Technology](https://mlsanity.com/paper/2311.00240/) (2023)

## Built on by (most cited first, 3 of 3)

- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [Hiding in the AI Traffic: Abusing MCP for LLM-Powered Agentic Red Teaming](https://mlsanity.com/paper/2511.15998/) (2025)
- [Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives](https://mlsanity.com/paper/2609.16694/) (2026)

## Abstract (arXiv)

Artificial Intelligence (AI) has witnessed major growth and integration across various domains. As AI systems become increasingly prevalent, they also become targets for threat actors to manipulate their functionality for malicious purposes. This survey paper on offensive AI will comprehensively cover various aspects related to attacks against and using AI systems. It will delve into the impact of offensive AI practices on different domains, including consumer, enterprise, and public digital infrastructure. The paper will explore adversarial machine learning, attacks against AI models, infrastructure, and interfaces, along with offensive techniques like information gathering, social engineering, and weaponized AI. Additionally, it will discuss the consequences and implications of offensive AI, presenting case studies, insights, and avenues for further research.