# ACDZero: MCTS Agent for Mastering Automated Cyber Defense

arXiv: https://arxiv.org/abs/2601.02196 · HTML: https://mlsanity.com/paper/2601.02196/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 5 Jan 2026.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://mlsanity.com/paper/2009.08120/) (2020)
- [SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation](https://mlsanity.com/paper/2307.01646/) (2023)
- [On Autonomous Agents in a Cyber Defence Environment](https://mlsanity.com/paper/2309.07388/) (2023)
- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [InSPO: Unlocking Intrinsic Self-Reflection for LLM Preference Optimization](https://mlsanity.com/paper/2512.23126/) (2025)
- [A Multiagent CyberBattleSim for RL Cyber Operation Agents](https://mlsanity.com/paper/2304.11052/) (2023)
- [Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers](https://mlsanity.com/paper/2304.14918/) (2023)
- [A Neurosymbolic Framework for Interpretable Cognitive Attack Detection in Augmented Reality](https://mlsanity.com/paper/2508.09185/) (2025)
- [Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents](https://mlsanity.com/paper/2509.16151/) (2025)
- [MALinZero: Efficient Low-Dimensional Search for Mastering Complex Multi-Agent Planning](https://mlsanity.com/paper/2511.06142/) (2025)
- [Mixed Text Recognition with Efficient Parameter Fine-Tuning and Transformer](https://mlsanity.com/paper/2404.12734/) (2024)
- [Perception Graph for Cognitive Attack Reasoning in Augmented Reality](https://mlsanity.com/paper/2509.05324/) (2025)
- [Calibrating and Rotating: A Unified Framework for Weight Conditioning in PEFT](https://mlsanity.com/paper/2511.00051/) (2025)

## Built on by (most cited first, 2 of 2)

- [MINT: Minimal Information Neuro-Symbolic Tree for Objective-Driven Knowledge-Gap Reasoning and Active Elicitation](https://mlsanity.com/paper/2602.05048/) (2026)
- [CyberWorld: World Models for Sample-Efficient Autonomous Cyber Defense](https://mlsanity.com/paper/2609.31893/) (2026)

## Abstract (arXiv)

Automated cyber defense (ACD) seeks to protect computer networks with minimal or no human intervention, reacting to intrusions by taking corrective actions such as isolating hosts, resetting services, deploying decoys, or updating access controls. However, existing approaches for ACD, such as deep reinforcement learning (RL), often face difficult exploration in complex networks with large decision/state spaces and thus require an expensive amount of samples. Inspired by the need to learn sample-efficient defense policies, we frame ACD in CAGE Challenge 4 (CAGE-4 / CC4) as a context-based partially observable Markov decision problem and propose a planning-centric defense policy based on Monte Carlo Tree Search (MCTS). It explicitly models the exploration-exploitation tradeoff in ACD and uses statistical sampling to guide exploration and decision making. We make novel use of graph neural networks (GNNs) to embed observations from the network as attributed graphs, to enable permutation-invariant reasoning over hosts and their relationships. To make our solution practical in complex search spaces, we guide MCTS with learned graph embeddings and priors over graph-edit actions, combining model-free generalization and policy distillation with look-ahead planning. We evaluate the resulting agent on CC4 scenarios involving diverse network structures and adversary behaviors, and show that our search-guided, graph-embedding-based planning improves defense reward and robustness relative to state-of-the-art RL baselines.