# Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach

arXiv: https://arxiv.org/abs/1812.09808 · HTML: https://mlsanity.com/paper/1812.09808/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2018.

## Built on by (most cited first, 29 of 29)

- [Distributionally Robust Learning](https://mlsanity.com/paper/2108.08993/) (2021)
- [Sinkhorn Distributionally Robust Optimization](https://mlsanity.com/paper/2109.11926/) (2021)
- [A Unified Wasserstein Distributional Robustness Framework for Adversarial Training](https://mlsanity.com/paper/2202.13437/) (2022)
- [A Finite Sample Complexity Bound for Distributionally Robust Q-learning](https://mlsanity.com/paper/2302.13203/) (2023)
- [Statistical Analysis of Wasserstein Distributionally Robust Estimators](https://mlsanity.com/paper/2108.02120/) (2021)
- [On the Foundation of Distributionally Robust Reinforcement Learning](https://mlsanity.com/paper/2311.09018/) (2023)
- [Single-Trajectory Distributionally Robust Reinforcement Learning](https://mlsanity.com/paper/2301.11721/) (2023)
- [Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control](https://mlsanity.com/paper/2109.08947/) (2021)
- [Learning High Dimensional Wasserstein Geodesics](https://mlsanity.com/paper/2102.02992/) (2021)
- [Nash Equilibria, Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization](https://mlsanity.com/paper/2303.03900/) (2023)
- [Optimal Transport Model Distributional Robustness](https://mlsanity.com/paper/2306.04178/) (2023)
- [Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion](https://mlsanity.com/paper/2310.16546/) (2023)
- [Wasserstein Distributionally Robust Control Barrier Function using Conditional Value-at-Risk with Differentiable Convex Programming](https://mlsanity.com/paper/2309.08700/) (2023)
- [Sequential Decision-Making under Uncertainty: A Robust MDPs review](https://mlsanity.com/paper/2404.00940/) (2024)
- [Distributionally Robust Policy and Lyapunov-Certificate Learning](https://mlsanity.com/paper/2404.03017/) (2024)
- [Policy Evaluation in Distributional LQR](https://mlsanity.com/paper/2303.13657/) (2023)
- [Adjusted Wasserstein Distributionally Robust Estimator in Statistical Learning](https://mlsanity.com/paper/2303.15579/) (2023)
- [Distributionally Safe Reinforcement Learning under Model Uncertainty: A Single-Level Approach by Differentiable Convex Programming](https://mlsanity.com/paper/2310.02459/) (2023)
- [Near-Optimal Sample Complexities of Divergence-based S-rectangular Distributionally Robust Reinforcement Learning](https://mlsanity.com/paper/2505.12202/) (2025)
- [Distributionally Robust Deep Q-Learning](https://mlsanity.com/paper/2505.19058/) (2025)
- [Risk-Averse MDPs under Reward Ambiguity](https://mlsanity.com/paper/2301.01045/) (2023)
- [Propagating Kernel Ambiguity Sets in Nonlinear Data-driven Dynamics Models](https://mlsanity.com/paper/2304.14057/) (2023)
- [Learning Optimal Distributionally Robust Stochastic Control in Continuous State Spaces](https://mlsanity.com/paper/2406.11281/) (2024)
- [Distributionally Robust Control with End-to-End Statistically Guaranteed Metric Learning](https://mlsanity.com/paper/2510.10214/) (2025)
- [Distributionally Robust Regret Optimal Control Under Moment-Based Ambiguity Sets](https://mlsanity.com/paper/2512.10906/) (2025)
- [Data-driven robust Markov decision processes on Borel spaces: performance guarantees via an axiomatic approach](https://mlsanity.com/paper/2603.08979/) (2026)
- [Policy Gradient Learning for Distributionally Robust Markov Decision Processes under Wasserstein Ambiguity](https://mlsanity.com/paper/2606.27610/) (2026)
- [Model-Based Reinforcement Learning for Heterogeneous Multi-Robot Task Assignment Under Distribution Shifts](https://mlsanity.com/paper/2608.21554/) (2026)
- [A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification](https://mlsanity.com/paper/2608.29789/) (2026)

## Abstract (arXiv)

Standard stochastic control methods assume that the probability distribution of uncertain variables is available. Unfortunately, in practice, obtaining accurate distribution information is a challenging task. To resolve this issue, we investigate the problem of designing a control policy that is robust against errors in the empirical distribution obtained from data. This problem can be formulated as a two-player zero-sum dynamic game problem, where the action space of the adversarial player is a Wasserstein ball centered at the empirical distribution. We propose computationally tractable value and policy iteration algorithms with explicit estimates of the number of iterations required for constructing an ε-optimal policy. We show that the contraction property of associated Bellman operators extends a single-stage out-of-sample performance guarantee, obtained using a measure concentration inequality, to the corresponding multi-stage guarantee without any degradation in the confidence level. In addition, we characterize an explicit form of the optimal distributionally robust control policy and the worst-case distribution policy for linear-quadratic problems with Wasserstein penalty. Our study indicates that dynamic programming and Kantorovich duality play a critical role in solving and analyzing the Wasserstein distributionally robust stochastic control problems.