# Lessons from Penetration Tests on Large-Scale Agent Systems

arXiv: https://arxiv.org/abs/2605.27042 · HTML: https://mlsanity.com/paper/2605.27042/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 12 references that are in the map.
- First posted on arXiv 26 May 2026.

## Builds on (most cited first, 1 of 1)

- [Systems Security Foundations for Agentic Computing](https://mlsanity.com/paper/2512.01295/) (2025)

## Built on by (most cited first, 1 of 1)

- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)

## Abstract (arXiv)

As AI systems gain increasing autonomy and execution capability, the number of discovered security vulnerabilities continues to rise. However, many of these vulnerabilities are not fundamentally novel, but instead reflect recurring classes of weaknesses long observed in prior computing systems. Execution-capable AI agents are effectively unbounded, self-modifying programs that interact extensively with multiple layers of the computing stack. This broad interaction surface imposes a significant security burden on developers, who must reason about and secure complex cross-layer behaviors. Prior research has primarily focused on vulnerabilities in open-source agents and agent frameworks. In contrast, it remains unclear whether proprietary agent systems -- developed under stricter coding standards and formal review processes -- exhibit similar security weaknesses. In this paper, we present findings from two penetration tests conducted in 2025 against proprietary agent products and evaluate whether the security posture of AI agents has improved since these assessments.