# Discovering Command and Control (C2) Channels on Tor and Public Networks Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2402.09200 · HTML: https://mlsanity.com/paper/2402.09200/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Safe and Efficient Off-Policy Reinforcement Learning](https://mlsanity.com/paper/1606.02647/) (2016)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Deep hierarchical reinforcement agents for automated penetration testing](https://mlsanity.com/paper/2109.06449/) (2021)
- [Using Cyber Terrain in Reinforcement Learning for Penetration Testing](https://mlsanity.com/paper/2108.07124/) (2021)
- [Crown Jewels Analysis using Reinforcement Learning with Attack Graphs](https://mlsanity.com/paper/2108.09358/) (2021)
- [Discovering Exfiltration Paths Using Reinforcement Learning with Attack Graphs](https://mlsanity.com/paper/2201.12416/) (2022)
- [A Layered Reference Model for Penetration Testing with Reinforcement Learning and Attack Graphs](https://mlsanity.com/paper/2206.06934/) (2022)
- [Exposing Surveillance Detection Routes via Reinforcement Learning, Attack Graphs, and Cyber Terrain](https://mlsanity.com/paper/2211.03027/) (2022)
- [Discovering Command and Control Channels Using Reinforcement Learning](https://mlsanity.com/paper/2401.07154/) (2024)
- [Command & Control: Understanding, Denying and Detecting - A review of malware C2 techniques, detection and defences](https://mlsanity.com/paper/1408.1136/) (2014)

## Abstract (arXiv)

Command and control (C2) channels are an essential component of many types of cyber attacks, as they enable attackers to remotely control their malware-infected machines and execute harmful actions, such as propagating malicious code across networks, exfiltrating confidential data, or initiating distributed denial of service (DDoS) attacks. Identifying these C2 channels is therefore crucial in helping to mitigate and prevent cyber attacks. However, identifying C2 channels typically involves a manual process, requiring deep knowledge and expertise in cyber operations. In this paper, we propose a reinforcement learning (RL) based approach to automatically emulate C2 attack campaigns using both the normal (public) and the Tor networks. In addition, payload size and network firewalls are configured to simulate real-world attack scenarios. Results on a typical network configuration show that the RL agent can automatically discover resilient C2 attack paths utilizing both Tor-based and conventional communication channels, while also bypassing network firewalls.