# Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models

arXiv: https://arxiv.org/abs/1904.01068 · HTML: https://mlsanity.com/paper/1904.01068/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2019.

## Built on by (most cited first, 7 of 7)

- [Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art](https://mlsanity.com/paper/2101.09505/) (2021)
- [Safe Policy Optimization with Local Generalized Linear Function Approximations](https://mlsanity.com/paper/2111.04894/) (2021)
- [Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Function](https://mlsanity.com/paper/2108.02307/) (2021)
- [DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning](https://mlsanity.com/paper/2402.15957/) (2024)
- [Safe Exploration Method for Reinforcement Learning under Existence of Disturbance](https://mlsanity.com/paper/2209.15452/) (2022)
- [Are Evolutionary Algorithms Safe Optimizers?](https://mlsanity.com/paper/2203.12622/) (2022)
- [Safe Sequential Optimization for Switching Environments](https://mlsanity.com/paper/2311.02119/) (2023)

## Abstract (arXiv)

We propose a safe exploration algorithm for deterministic Markov Decision Processes with unknown transition models. Our algorithm guarantees safety by leveraging Lipschitz-continuity to ensure that no unsafe states are visited during exploration. Unlike many other existing techniques, the provided safety guarantee is deterministic. Our algorithm is optimized to reduce the number of actions needed for exploring the safe space. We demonstrate the performance of our algorithm in comparison with baseline methods in simulation on navigation tasks.