# Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agents

arXiv: https://arxiv.org/abs/2412.02875 · HTML: https://mlsanity.com/paper/2412.02875/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 14 of 14)

- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?](https://mlsanity.com/paper/2006.14911/) (2020)
- [Behavior Trees in Robotics and AI: An Introduction](https://mlsanity.com/paper/1709.00084/) (2017)
- [Full-Spectrum Out-of-Distribution Detection](https://mlsanity.com/paper/2204.05306/) (2022)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systems](https://mlsanity.com/paper/2001.10494/) (2020)
- [Network Environment Design for Autonomous Cyberdefense](https://mlsanity.com/paper/2103.07583/) (2021)
- [On Autonomous Agents in a Cyber Defence Environment](https://mlsanity.com/paper/2309.07388/) (2023)
- [Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies](https://mlsanity.com/paper/2211.15557/) (2022)
- [Efficient Out-of-Distribution Detection Using Latent Space of β-VAE for Cyber-Physical Systems](https://mlsanity.com/paper/2108.11800/) (2021)
- [Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection](https://mlsanity.com/paper/2404.07099/) (2024)
- [Autonomous Network Defence using Reinforcement Learning](https://mlsanity.com/paper/2409.18197/) (2024)
- [Unified Out-Of-Distribution Detection: A Model-Specific Perspective](https://mlsanity.com/paper/2304.06813/) (2023)
- [Designing Robust Cyber-Defense Agents with Evolving Behavior Trees](https://mlsanity.com/paper/2410.16383/) (2024)

## Built on by (most cited first, 3 of 3)

- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06921/) (2025)
- [Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents](https://mlsanity.com/paper/2606.18223/) (2026)

## Abstract (arXiv)

Autonomous agents for cyber applications take advantage of modern defense techniques by adopting intelligent agents with conventional and learning-enabled components. These intelligent agents are trained via reinforcement learning (RL) algorithms, and can learn, adapt to, reason about and deploy security rules to defend networked computer systems while maintaining critical operational workflows. However, the knowledge available during training about the state of the operational network and its environment may be limited. The agents should be trustworthy so that they can reliably detect situations they cannot handle, and hand them over to cyber experts. In this work, we develop an out-of-distribution (OOD) Monitoring algorithm that uses a Probabilistic Neural Network (PNN) to detect anomalous or OOD situations of RL-based agents with discrete states and discrete actions. To demonstrate the effectiveness of the proposed approach, we integrate the OOD monitoring algorithm with a neurosymbolic autonomous cyber agent that uses behavior trees with learning-enabled components. We evaluate the proposed approach in a simulated cyber environment under different adversarial strategies. Experimental results over a large number of episodes illustrate the overall efficiency of our proposed approach.