# Beyond Predictable Paths: Redefining AI Security Incident Reporting for Agents

arXiv: https://arxiv.org/abs/2609.24515 · HTML: https://mlsanity.com/paper/2609.24515/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database](https://mlsanity.com/paper/2011.08512/) (2020)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences](https://mlsanity.com/paper/2206.08451/) (2022)
- [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025)
- [EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System](https://mlsanity.com/paper/2509.10540/) (2025)
- [AI Risk Management Should Incorporate Both Safety and Security](https://mlsanity.com/paper/2405.19524/) (2024)
- [Trojan Hippo: Weaponizing Agent Memory for Data Exfiltration](https://mlsanity.com/paper/2605.01970/) (2026)
- [Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluation](https://mlsanity.com/paper/2511.03841/) (2025)
- [Visualizing Privacy-Utility Trade-Offs in Differentially Private Data Releases](https://mlsanity.com/paper/2201.05964/) (2022)
- [Designing Incident Reporting Systems for Harms from General-Purpose AI](https://mlsanity.com/paper/2511.05914/) (2025)
- [Energy-Latency Attacks: A New Adversarial Threat to Deep Learning](https://mlsanity.com/paper/2503.04963/) (2025)
- [Agent Security is a Systems Problem](https://mlsanity.com/paper/2605.18991/) (2026)
- [Web Agents Should Adopt the Plan-Then-Execute Paradigm](https://mlsanity.com/paper/2605.14290/) (2026)
- [Toward Comprehensive Risk Assessments and Assurance of AI-Based Systems](https://mlsanity.com/paper/2606.29390/) (2026)
- [Differentially Private Sparse Vectors with Low Error, Optimal Space, and Fast Access](https://mlsanity.com/paper/2106.10068/) (2021)
- [Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety](https://mlsanity.com/paper/2605.22643/) (2026)
- [MAGIQ: A Post-Quantum Multi-Agentic AI Governance System with Provable Security](https://mlsanity.com/paper/2605.06933/) (2026)
- [Establishing Minimum Elements for Effective Vulnerability Management in AI Software](https://mlsanity.com/paper/2411.11317/) (2024)

## Abstract (arXiv)

AI agents are being deployed rapidly, accompanied by a growing number of AI-specific attacks and corresponding incidents. As incident reporting becomes increasingly important for legal compliance, governance, accountability, and security; current frameworks must be adapted to the unique characteristics of AI agents. In this paper, two editorial authors compare AI systems and AI agents and, drawing on input from 23 experts in academia and industry, identify the information required for reporting incidents where the security of AI agents is harmed. %involving AI agents. Potential reporting elements include, for example, agent memory and memory accesses, actual and potential levels of autonomy, and tool usage. Based on these findings, we identify several open research questions, including how to efficiently record incidents and how to determine whether vulnerabilities and incidents generalize. Expert feedback also highlighted potential reporting weaknesses, such as risks of data leakage and attacks targeting the reporting infrastructure itself, creating additional research needs. Lastly, we summarize privacy requirements and outline research directions for the secure and trustworthy deployment of AI agents.