# Red Team Redemption: A Structured Comparison of Open-Source Tools for Adversary Emulation

arXiv: https://arxiv.org/abs/2408.15645 · HTML: https://mlsanity.com/paper/2408.15645/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 2 of its 28 references that are in the map.
- First posted on arXiv 28 Aug 2024.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 2 of 2)

- [SoK: A Survey of Open-Source Threat Emulators](https://mlsanity.com/paper/2003.01518/) (2020)
- [Laccolith: Hypervisor-Based Adversary Emulation with Anti-Detection](https://mlsanity.com/paper/2311.08274/) (2023)

## Built on by (most cited first, 2 of 2)

- [Simulating Cyberattacks through a Breach Attack Simulation (BAS) Platform empowered by Security Chaos Engineering (SCE)](https://mlsanity.com/paper/2508.03882/) (2025)
- [A Cyber Range Evaluation of Autonomous Network Incident Response Agents](https://mlsanity.com/paper/2609.16541/) (2026)

## Abstract (arXiv)

Red teams simulate adversaries and conduct sophisticated attacks against defenders without informing them about used tactics in advance. These interactive cyber exercises are highly beneficial to assess and improve the security posture of organizations, detect vulnerabilities, and train employees. Unfortunately, they are also time-consuming and expensive, which often limits their scale or prevents them entirely. To address this situation, adversary emulation tools partially automate attacker behavior and enable fast, continuous, and repeatable security testing even when involved personnel lacks red teaming experience. Currently, a wide range of tools designed for specific use-cases and requirements exist. To obtain an overview of these solutions, we conduct a review and structured comparison of nine open-source adversary emulation tools. To this end, we assemble a questionnaire with 80 questions addressing relevant aspects, including setup, support, documentation, usability, and technical features. In addition, we conduct a user study with domain experts to investigate the importance of these aspects for distinct user roles. Based on the evaluation and user feedback, we rank the tools and find MITRE Caldera, Metasploit, and Atomic Red Team on top.