# A Factored MDP Approach To Moving Target Defense With Dynamic Threat Modeling and Cost Efficiency

arXiv: https://arxiv.org/abs/2408.08934 · HTML: https://mlsanity.com/paper/2408.08934/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 34 references that are in the map.
- First posted on arXiv 16 Aug 2024.

## Builds on (most cited first, 9 of 9)

- [Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mapping](https://mlsanity.com/paper/2006.13165/) (2020)
- [Nearly Minimax Optimal Reinforcement Learning for Discounted MDPs](https://mlsanity.com/paper/2010.00587/) (2020)
- [Regret Bounds for Discounted MDPs](https://mlsanity.com/paper/2002.05138/) (2020)
- [Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense](https://mlsanity.com/paper/2007.10457/) (2020)
- [A Survey on Device Behavior Fingerprinting: Data Sources, Techniques, Application Scenarios, and Datasets](https://mlsanity.com/paper/2008.03343/) (2020)
- [Toward Proactive, Adaptive Defense: A Survey on Moving Target Defense](https://mlsanity.com/paper/1909.08092/) (2019)
- [Technical Aspects of Cyber Kill Chain](https://mlsanity.com/paper/1606.03184/) (2016)
- [Spatial-Temporal Moving Target Defense: A Markov Stackelberg Game Model](https://mlsanity.com/paper/2002.10390/) (2020)
- [Adaptive MTD Security using Markov Game Modeling](https://mlsanity.com/paper/1811.00651/) (2018)

## Abstract (arXiv)

Moving Target Defense (MTD) has emerged as a proactive and dynamic framework to counteract evolving cyber threats. Traditional MTD approaches often rely on assumptions about the attackers knowledge and behavior. However, real-world scenarios are inherently more complex, with adaptive attackers and limited prior knowledge of their payoffs and intentions. This paper introduces a novel approach to MTD using a Markov Decision Process (MDP) model that does not rely on predefined attacker payoffs. Our framework integrates the attackers real-time responses into the defenders MDP using a dynamic Bayesian Network. By employing a factored MDP model, we provide a comprehensive and realistic system representation. We also incorporate incremental updates to an attack response predictor as new data emerges. This ensures an adaptive and robust defense mechanism. Additionally, we consider the costs of switching configurations in MTD, integrating them into the reward structure to balance execution and defense costs. We first highlight the challenges of the problem through a theoretical negative result on regret. However, empirical evaluations demonstrate the frameworks effectiveness in scenarios marked by high uncertainty and dynamically changing attack landscapes.