# Deterministic Sequencing of Exploration and Exploitation for Reinforcement Learning

arXiv: https://arxiv.org/abs/2209.05408 · HTML: https://mlsanity.com/paper/2209.05408/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 42 references that are in the map.
- First posted on arXiv 12 Sep 2022.

## Builds on (most cited first, 5 of 5)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [On Regret-Optimal Learning in Decentralized Multi-player Multi-armed Bandits](https://mlsanity.com/paper/1505.00553/) (2015)
- [On Abruptly-Changing and Slowly-Varying Multiarmed Bandit Problems](https://mlsanity.com/paper/1802.08380/) (2018)
- [Multi-Robot Gaussian Process Estimation and Coverage: A Deterministic Sequencing Algorithm and Regret Analysis](https://mlsanity.com/paper/2101.04306/) (2021)
- [Structural Properties of Optimal Fidelity Selection Policies for Human-in-the-loop Queues](https://mlsanity.com/paper/2201.09990/) (2022)

## Built on by (most cited first, 1 of 1)

- [Interaction-Aware Trajectory Planning for Autonomous Vehicles with Analytic Integration of Neural Networks into Model Predictive Control](https://mlsanity.com/paper/2301.05393/) (2023)

## Abstract (arXiv)

We propose Deterministic Sequencing of Exploration and Exploitation (DSEE) algorithm with interleaving exploration and exploitation epochs for model-based RL problems that aim to simultaneously learn the system model, i.e., a Markov decision process (MDP), and the associated optimal policy. During exploration, DSEE explores the environment and updates the estimates for expected reward and transition probabilities. During exploitation, the latest estimates of the expected reward and transition probabilities are used to obtain a robust policy with high probability. We design the lengths of the exploration and exploitation epochs such that the cumulative regret grows as a sub-linear function of time.