# Bounding the Difference between the Values of Robust and Non-Robust Markov Decision Problems

arXiv: https://arxiv.org/abs/2308.05520 · HTML: https://mlsanity.com/paper/2308.05520/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2023.

## Built on by (most cited first, 3 of 3)

- [Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data](https://mlsanity.com/paper/2505.23062/) (2025)
- [Q-Learning under Finite Model Uncertainty](https://mlsanity.com/paper/2407.04259/) (2024)
- [Data-driven robust Markov decision processes on Borel spaces: performance guarantees via an axiomatic approach](https://mlsanity.com/paper/2603.08979/) (2026)

## Abstract (arXiv)

In this note we provide an upper bound for the difference between the value function of a distributionally robust Markov decision problem and the value function of a non-robust Markov decision problem, where the ambiguity set of probability kernels of the distributionally robust Markov decision process is described by a Wasserstein-ball around some reference kernel whereas the non-robust Markov decision process behaves according to a fixed probability kernel contained in the ambiguity set. Our derived upper bound for the difference between the value functions is dimension-free and depends linearly on the radius of the Wasserstein-ball.