mlsanityData to 7 Oct 2026

Beyond Predictable Paths: Redefining AI Security Incident Reporting for Agents

arXiv 2609.24515 ↗First posted 21 Sep 2026cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

AI agents are being deployed rapidly, accompanied by a growing number of AI-specific attacks and corresponding incidents. As incident reporting becomes increasingly important for legal compliance, governance, accountability, and security; current frameworks must be adapted to the unique characteristics of AI agents. In this paper, two editorial authors compare AI systems and AI agents and, drawing on input from 23 experts in academia and industry, identify the information required for reporting incidents where the security of AI agents is harmed. %involving AI agents. Potential reporting elements include, for example, agent memory and memory accesses, actual and potential levels of autonomy, and tool usage. Based on these findings, we identify several open research questions, including how to efficiently record incidents and how to determine whether vulnerabilities and incidents generalize. Expert feedback also highlighted potential reporting weaknesses, such as risks of data leakage and attacks targeting the reporting infrastructure itself, creating additional research needs. Lastly, we summarize privacy requirements and outline research directions for the secure and trustworthy deployment of AI agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 66 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

23 of 66 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  3. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  4. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  5. 2025Multi-Agent Risks from Advanced AIcited by 150
  6. 2020Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Databasecited by 95
  7. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 93
  8. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  9. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  10. 2025EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM Systemcited by 26
  11. 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
  12. 2026Trojan Hippo: Weaponizing Agent Memory for Data Exfiltrationcited by 12
11 more
  1. 2022Visualizing Privacy-Utility Trade-Offs in Differentially Private Data Releasescited by 7
  2. 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluationcited by 6
  3. 2025Designing Incident Reporting Systems for Harms from General-Purpose AIcited by 6
  4. 2025Energy-Latency Attacks: A New Adversarial Threat to Deep Learningcited by 5
  5. 2026Toward Comprehensive Risk Assessments and Assurance of AI-Based Systemscited by 4
  6. 2021Differentially Private Sparse Vectors with Low Error, Optimal Space, and Fast Accesscited by 3
  7. 2026Web Agents Should Adopt the Plan-Then-Execute Paradigmcited by 3
  8. 2026Boiling the Frog: A Multi-Turn Benchmark for Agentic Safetycited by 3
  9. 2026MAGIQ: A Post-Quantum Multi-Agentic AI Governance System with Provable Securitycited by 2
  10. 2026Agent Security is a Systems Problemcited by 2
  11. 2024Establishing Minimum Elements for Effective Vulnerability Management in AI Softwarecited by 1

Built on by

Not cited yet.