# Surveillance Evasion Through Bayesian Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.14811 · HTML: https://mlsanity.com/paper/2109.14811/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 37 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 10 of 10)

- [Minimax Regret Bounds for Reinforcement Learning](https://mlsanity.com/paper/1703.05449/) (2017)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning](https://mlsanity.com/paper/1510.08906/) (2015)
- [Learning Adversarial MDPs with Bandit Feedback and Unknown Transition](https://mlsanity.com/paper/1912.01192/) (2019)
- [Scalable Log Determinants for Gaussian Process Kernel Learning](https://mlsanity.com/paper/1711.03481/) (2017)
- [Deterministic control of randomly-terminated processes](https://mlsanity.com/paper/1310.7161/) (2013)
- [Time-Dependent Surveillance-Evasion Games](https://mlsanity.com/paper/1903.01332/) (2019)
- [Control-Theoretic Models of Environmental Crime](https://mlsanity.com/paper/1906.09289/) (2019)
- [Optimality and robustness in path-planning under initial uncertainty](https://mlsanity.com/paper/2106.11405/) (2021)

## Abstract (arXiv)

We consider a task of surveillance-evading path-planning in a continuous setting. An Evader strives to escape from a 2D domain while minimizing the risk of detection (and immediate capture). The probability of detection is path-dependent and determined by the spatially inhomogeneous surveillance intensity, which is fixed but a priori unknown and gradually learned in the multi-episodic setting. We introduce a Bayesian reinforcement learning algorithm that relies on a Gaussian Process regression (to model the surveillance intensity function based on the information from prior episodes), numerical methods for Hamilton-Jacobi PDEs (to plan the best continuous trajectories based on the current model), and Confidence Bounds (to balance the exploration vs exploitation). We use numerical experiments and regret metrics to highlight the significant advantages of our approach compared to traditional graph-based algorithms of reinforcement learning.