# CyberForce: A Federated Reinforcement Learning Framework for Malware Mitigation

arXiv: https://arxiv.org/abs/2308.05978 · HTML: https://mlsanity.com/paper/2308.05978/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 11 Aug 2023.

## Builds on (most cited first, 10 of 10)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018)
- [Federated Learning for Malware Detection in IoT Devices](https://mlsanity.com/paper/2104.09994/) (2021)
- [Reinforcement Learning for IoT Security: A Comprehensive Survey](https://mlsanity.com/paper/2102.07247/) (2021)
- [Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense](https://mlsanity.com/paper/2007.10457/) (2020)
- [A Survey on Device Behavior Fingerprinting: Data Sources, Techniques, Application Scenarios, and Datasets](https://mlsanity.com/paper/2008.03343/) (2020)
- [Electrosense: Open and Big Spectrum Data](https://mlsanity.com/paper/1703.09989/) (2017)
- [RL and Fingerprinting to Select Moving Target Defense Mechanisms for Zero-day Attacks in IoT](https://mlsanity.com/paper/2212.14647/) (2022)
- [A Lightweight Moving Target Defense Framework for Multi-purpose Malware Affecting IoT Devices](https://mlsanity.com/paper/2210.07719/) (2022)
- [Mitigating Communications Threats in Decentralized Federated Learning through Moving Target Defense](https://mlsanity.com/paper/2307.11730/) (2023)

## Built on by (most cited first, 1 of 1)

- [Can Artificial Intelligence Embody Moral Values?](https://mlsanity.com/paper/2408.12250/) (2024)

## Abstract (arXiv)

Recent research has shown that the integration of Reinforcement Learning (RL) with Moving Target Defense (MTD) can enhance cybersecurity in Internet-of-Things (IoT) devices. Nevertheless, the practicality of existing work is hindered by data privacy concerns associated with centralized data processing in RL, and the unsatisfactory time needed to learn right MTD techniques that are effective against a rising number of heterogeneous zero-day attacks. Thus, this work presents CyberForce, a framework that combines Federated and Reinforcement Learning (FRL) to collaboratively and privately learn suitable MTD techniques for mitigating zero-day attacks. CyberForce integrates device fingerprinting and anomaly detection to reward or penalize MTD mechanisms chosen by an FRL-based agent. The framework has been deployed and evaluated in a scenario consisting of ten physical devices of a real IoT platform affected by heterogeneous malware samples. A pool of experiments has demonstrated that CyberForce learns the MTD technique mitigating each attack faster than existing RL-based centralized approaches. In addition, when various devices are exposed to different attacks, CyberForce benefits from knowledge transfer, leading to enhanced performance and reduced learning time in comparison to recent works. Finally, different aggregation algorithms used during the agent learning process provide CyberForce with notable robustness to malicious attacks.