mlsanityData to 14 Oct 2026

Towards Production-Worthy Simulation for Autonomous Cyber Operations

arXiv 2508.19278 ↗First posted 23 Aug 2025cs.CR cs.AI cs.LGcited by 2

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Simulated environments have proven invaluable in Autonomous Cyber Operations (ACO) where Reinforcement Learning (RL) agents can be trained without the computational overhead of emulation. These environments must accurately represent cybersecurity scenarios while producing the necessary signals to support RL training. In this study, we present a framework where we first extend CybORG's Cage Challenge 2 environment by implementing three new actions: Patch, Isolate, and Unisolate, to better represent the capabilities available to human operators in real-world settings. We then propose a design for agent development where we modify the reward signals and the agent's feature space to enhance training performance. To validate these modifications, we train DQN and PPO agents in the updated environment. Our study demonstrates that CybORG can be extended with additional realistic functionality, while maintaining its ability to generate informative training signals for RL agents.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 16 references that are in the map. First posted on arXiv 23 Aug 2025.

Builds on

4 of 16 references.

References · 4

  1. 2023Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learningcited by 12
  2. 2024LLM Honeypot: Leveraging Large Language Models as Advanced Interactive Honeypot Systemscited by 10
  3. 2023Deep Reinforcement Learning for Cyber System Defense under Dynamic Adversarial Uncertaintiescited by 3
  4. 2024The Path To Autonomous Cyber Defensecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Large Language Model Integration with Reinforcement Learning to Augment Decision-Making in Autonomous Cyber Operationscited by 4
  2. 2026Distilling Knowledge from Large Language Models into Lightweight Reinforcement Learning Agents for Autonomous Cyber Operationscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2021CybORG: A Gym for the Development of Autonomous Cyber Agents
  2. 2025A Comparative Evaluation of Teacher-Guided Reinforcement Learning Techniques for Autonomous Cyber Operations
  3. 2023Unified Emulation-Simulation Training Environment for Autonomous Cyber Agents
  4. 2026Building Better Environments for Autonomous Cyber Defence
  5. 2024Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense
  6. 2024Exploring reinforcement learning for incident response in autonomous military vehicles
  7. 2024CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents
  8. 2026A Cyber Range Evaluation of Autonomous Network Incident Response Agents