# Robust Q-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty

arXiv: https://arxiv.org/abs/2210.00898 · HTML: https://mlsanity.com/paper/2210.00898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2022.

## Abstract (arXiv)

We present a novel Q-learning algorithm tailored to solve distributionally robust Markov decision problems where the corresponding ambiguity set of transition probabilities for the underlying Markov decision process is a Wasserstein ball around a (possibly estimated) reference measure. We prove convergence of the presented algorithm and provide several examples also using real data to illustrate both the tractability of our algorithm as well as the benefits of considering distributional robustness when solving stochastic optimal control problems, in particular when the estimated distributions turn out to be misspecified in practice.