# On Autonomous Agents in a Cyber Defence Environment

arXiv: https://arxiv.org/abs/2309.07388 · HTML: https://mlsanity.com/paper/2309.07388/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2023.

## Built on by (most cited first, 26 of 26)

- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2410.16324/) (2024)
- [Adversarial Inception Backdoor Attacks against Reinforcement Learning](https://mlsanity.com/paper/2410.13995/) (2024)
- [Deep Reinforcement Learning for Autonomous Cyber Defence: A Survey](https://mlsanity.com/paper/2310.07745/) (2023)
- [Designing Robust Cyber-Defense Agents with Evolving Behavior Trees](https://mlsanity.com/paper/2410.16383/) (2024)
- [Large Language Model Integration with Reinforcement Learning to Augment Decision-Making in Autonomous Cyber Operations](https://mlsanity.com/paper/2509.05311/) (2025)
- [Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agents](https://mlsanity.com/paper/2412.02875/) (2024)
- [Learning Optimal Defender Strategies for CAGE-2 using a POMDP Model](https://mlsanity.com/paper/2509.06539/) (2025)
- [Beyond Rewards in Reinforcement Learning for Cyber Defence](https://mlsanity.com/paper/2602.04809/) (2026)
- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [Interpreting Agent Behaviors in Reinforcement-Learning-Based Cyber-Battle Simulation Platforms](https://mlsanity.com/paper/2506.08192/) (2025)
- [Learning to Communicate in Multi-Agent Reinforcement Learning for Autonomous Cyber Defence](https://mlsanity.com/paper/2507.14658/) (2025)
- [ACDZero: MCTS Agent for Mastering Automated Cyber Defense](https://mlsanity.com/paper/2601.02196/) (2026)
- [Quantitative Resilience Modeling for Autonomous Cyber Defense](https://mlsanity.com/paper/2503.02780/) (2025)
- [Less is more? Rewards in RL for Cyber Defence](https://mlsanity.com/paper/2503.03245/) (2025)
- [General Autonomous Cybersecurity Defense: Learning Robust Policies for Dynamic Topologies and Diverse Attackers](https://mlsanity.com/paper/2506.22706/) (2025)
- [Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP](https://mlsanity.com/paper/2605.16205/) (2026)
- [FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast](https://mlsanity.com/paper/2605.16233/) (2026)
- [Distilling Knowledge from Large Language Models into Lightweight Reinforcement Learning Agents for Autonomous Cyber Operations](https://mlsanity.com/paper/2607.28826/) (2026)
- [Explore Reinforced: Equilibrium Approximation with Reinforcement Learning](https://mlsanity.com/paper/2412.02016/) (2024)
- [PoolFlip: A Multi-Agent Reinforcement Learning Security Environment for Cyber Defense](https://mlsanity.com/paper/2508.19488/) (2025)
- [Strategic Cyber Defense via Reinforcement Learning-Guided Combinatorial Auctions](https://mlsanity.com/paper/2509.10983/) (2025)
- [Design Principles for the Construction of a Benchmark Evaluating Security Operation Capabilities of Multi-agent AI Systems](https://mlsanity.com/paper/2603.28998/) (2026)
- [Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents](https://mlsanity.com/paper/2606.18223/) (2026)
- [Towards Hierarchical Cyber Defense with Large Language Models: From Planning to Execution](https://mlsanity.com/paper/2610.00590/) (2026)
- [Ask the Expert: LLM-Guided Reinforcement Learning for Autonomous Cyber Defense](https://mlsanity.com/paper/2610.09337/) (2026)

## Abstract (arXiv)

Autonomous Cyber Defence is required to respond to high-tempo cyber-attacks. To facilitate the research in this challenging area, we explore the utility of the autonomous cyber operation environments presented as part of the Cyber Autonomy Gym for Experimentation (CAGE) Challenges, with a specific focus on CAGE Challenge 2. CAGE Challenge 2 required a defensive Blue agent to defend a network from an attacking Red agent. We provide a detailed description of the this challenge and describe the approaches taken by challenge participants. From the submitted agents, we identify four classes of algorithms, namely, Single- Agent Deep Reinforcement Learning (DRL), Hierarchical DRL, Ensembles, and Non-DRL approaches. Of these classes, we found that the hierarchical DRL approach was the most capable of learning an effective cyber defensive strategy. Our analysis of the agent policies identified that different algorithms within the same class produced diverse strategies and that the strategy used by the defensive Blue agent varied depending on the strategy used by the offensive Red agent. We conclude that DRL algorithms are a suitable candidate for autonomous cyber defence applications.