# Cybersecurity AI: Hacking the AI Hackers via Prompt Injection

arXiv: https://arxiv.org/abs/2508.21669 · HTML: https://mlsanity.com/paper/2508.21669/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 3 of its 10 references that are in the map.
- First posted on arXiv 29 Aug 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 3 of 3)

- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Cybersecurity AI: The Dangerous Gap Between Automation and Autonomy](https://mlsanity.com/paper/2506.23592/) (2025)
- [Offensive Robot Cybersecurity](https://mlsanity.com/paper/2506.15343/) (2025)

## Built on by (most cited first, 2 of 2)

- [The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis](https://mlsanity.com/paper/2602.10453/) (2026)
- [Cybersecurity AI: Evaluating Agentic Cybersecurity in Attack/Defense CTFs](https://mlsanity.com/paper/2510.17521/) (2025)

## Abstract (arXiv)

We demonstrate how AI-powered cybersecurity tools can be turned against themselves through prompt injection attacks. Prompt injection is reminiscent of cross-site scripting (XSS): malicious text is hidden within seemingly trusted content, and when the system processes it, that text is transformed into unintended instructions. When AI agents designed to find and exploit vulnerabilities interact with malicious web servers, carefully crafted reponses can hijack their execution flow, potentially granting attackers system access. We present proof-of-concept exploits against the Cybersecurity AI (CAI) framework and its CLI tool, and detail our mitigations against such attacks in a multi-layered defense implementation. Our findings indicate that prompt injection is a recurring and systemic issue in LLM-based architectures, one that will require dedicated work to address, much as the security community has had to do with XSS in traditional web applications.