# Controlling a Markov Decision Process with an Abrupt Change in the Transition Kernel

arXiv: https://arxiv.org/abs/2210.04098 · HTML: https://mlsanity.com/paper/2210.04098/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2022.

## Built on by (most cited first, 2 of 2)

- [Predictive Control and Regret Analysis of Non-Stationary MDP with Look-ahead Information](https://mlsanity.com/paper/2409.08434/) (2024)
- [Reinforcement Learning with an Abrupt Model Change](https://mlsanity.com/paper/2304.11460/) (2023)

## Abstract (arXiv)

We consider the control of a Markov decision process (MDP) that undergoes an abrupt change in its transition kernel (mode). We formulate the problem of minimizing regret under control-switching based on mode change detection, compared to a mode-observing controller, as an optimal stopping problem. Using a sequence of approximations, we reduce it to a quickest change detection (QCD) problem with Markovian data, for which we characterize a state-dependent threshold-type optimal change detection policy. Numerical experiments illustrate various properties of our control-switching policy.