# Discovering Command and Control Channels Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2401.07154 · HTML: https://mlsanity.com/paper/2401.07154/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 13 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 7 of 7)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Using Cyber Terrain in Reinforcement Learning for Penetration Testing](https://mlsanity.com/paper/2108.07124/) (2021)
- [Crown Jewels Analysis using Reinforcement Learning with Attack Graphs](https://mlsanity.com/paper/2108.09358/) (2021)
- [Deep Q-Learning based Reinforcement Learning Approach for Network Intrusion Detection](https://mlsanity.com/paper/2111.13978/) (2021)
- [Discovering Exfiltration Paths Using Reinforcement Learning with Attack Graphs](https://mlsanity.com/paper/2201.12416/) (2022)

## Built on by (most cited first, 3 of 3)

- [Leveraging Reinforcement Learning in Red Teaming for Advanced Ransomware Attack Simulations](https://mlsanity.com/paper/2406.17576/) (2024)
- [Discovering Command and Control (C2) Channels on Tor and Public Networks Using Reinforcement Learning](https://mlsanity.com/paper/2402.09200/) (2024)
- [A Novel Reinforcement Learning Model for Post-Incident Malware Investigations](https://mlsanity.com/paper/2410.15028/) (2024)

## Abstract (arXiv)

Command and control (C2) paths for issuing commands to malware are sometimes the only indicators of its existence within networks. Identifying potential C2 channels is often a manually driven process that involves a deep understanding of cyber tradecraft. Efforts to improve discovery of these channels through using a reinforcement learning (RL) based approach that learns to automatically carry out C2 attack campaigns on large networks, where multiple defense layers are in place serves to drive efficiency for network operators. In this paper, we model C2 traffic flow as a three-stage process and formulate it as a Markov decision process (MDP) with the objective to maximize the number of valuable hosts whose data is exfiltrated. The approach also specifically models payload and defense mechanisms such as firewalls which is a novel contribution. The attack paths learned by the RL agent can in turn help the blue team identify high-priority vulnerabilities and develop improved defense strategies. The method is evaluated on a large network with more than a thousand hosts and the results demonstrate that the agent can effectively learn attack paths while avoiding firewalls.