# Adaptive Honeypot Engagement through Reinforcement Learning of Semi-Markov Decision Processes

arXiv: https://arxiv.org/abs/1906.12182 · HTML: https://mlsanity.com/paper/1906.12182/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2019.

## Built on by (most cited first, 9 of 9)

- [Reinforcement Learning for Feedback-Enabled Cyber Resilience](https://mlsanity.com/paper/2107.00783/) (2021)
- [Intrusion Prevention through Optimal Stopping](https://mlsanity.com/paper/2111.00289/) (2021)
- [Symbiotic Game and Foundation Models for Cyber Deception Operations in Strategic Cyber Warfare](https://mlsanity.com/paper/2403.10570/) (2024)
- [RADAMS: Resilient and Adaptive Alert and Attention Management Strategy against Informational Denial-of-Service (IDoS) Attacks](https://mlsanity.com/paper/2111.03463/) (2021)
- [The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2308.16562/) (2023)
- [Combating Informational Denial-of-Service (IDoS) Attacks: Modeling and Mitigation of Attentional Human Vulnerability](https://mlsanity.com/paper/2108.08255/) (2021)
- [RL-IoT: Reinforcement Learning to Interact with IoT Devices](https://mlsanity.com/paper/2105.00884/) (2021)
- [Design and Development of an Intelligent LLM-based LDAP Honeypot](https://mlsanity.com/paper/2509.16682/) (2025)
- [Towards Agentic Honeynet Configuration](https://mlsanity.com/paper/2603.14122/) (2026)

## Abstract (arXiv)

A honeynet is a promising active cyber defense mechanism. It reveals the fundamental Indicators of Compromise (IoCs) by luring attackers to conduct adversarial behaviors in a controlled and monitored environment. The active interaction at the honeynet brings a high reward but also introduces high implementation costs and risks of adversarial honeynet exploitation. In this work, we apply infinite-horizon Semi-Markov Decision Process (SMDP) to characterize a stochastic transition and sojourn time of attackers in the honeynet and quantify the reward-risk trade-off. In particular, we design adaptive long-term engagement policies shown to be risk-averse, cost-effective, and time-efficient. Numerical results have demonstrated that our adaptive engagement policies can quickly attract attackers to the target honeypot and engage them for a sufficiently long period to obtain worthy threat information. Meanwhile, the penetration probability is kept at a low level. The results show that the expected utility is robust against attackers of a large range of persistence and intelligence. Finally, we apply reinforcement learning to the SMDP to solve the curse of modeling. Under a prudent choice of the learning rate and exploration policy, we achieve a quick and robust convergence of the optimal policy and value.