# Distributionally Robust Learning

arXiv: https://arxiv.org/abs/2108.08993 · HTML: https://mlsanity.com/paper/2108.08993/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 188 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 30 of 44)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Unlabeled Data Improves Adversarial Robustness](https://mlsanity.com/paper/1905.13736/) (2019)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits](https://mlsanity.com/paper/1402.0555/) (2014)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Are Labels Required for Improving Adversarial Robustness?](https://mlsanity.com/paper/1905.13725/) (2019)
- [From Predictive to Prescriptive Analytics](https://mlsanity.com/paper/1402.5481/) (2014)
- [Adversarial Training Can Hurt Generalization](https://mlsanity.com/paper/1906.06032/) (2019)
- [Distributionally Robust Logistic Regression](https://mlsanity.com/paper/1509.09259/) (2015)
- [Regularization via Mass Transportation](https://mlsanity.com/paper/1710.10016/) (2017)
- [Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions](https://mlsanity.com/paper/1806.08141/) (2018)
- [Adversarially Robust Generalization Just Requires More Unlabeled Data](https://mlsanity.com/paper/1906.00555/) (2019)
- [A Wasserstein-type distance in the space of Gaussian Mixture Models](https://mlsanity.com/paper/1907.05254/) (2019)
- [Robustness to Adversarial Perturbations in Learning from Incomplete Data](https://mlsanity.com/paper/1905.13021/) (2019)
- [Distributional Robustness and Regularization in Reinforcement Learning](https://mlsanity.com/paper/2003.02894/) (2020)
- [Does Distributionally Robust Supervised Learning Give Robust Classifiers?](https://mlsanity.com/paper/1611.02041/) (2016)
- [Data-driven Inverse Optimization with Imperfect Information](https://mlsanity.com/paper/1512.05489/) (2015)
- [Data-Driven Estimation in Equilibrium Using Inverse Optimization](https://mlsanity.com/paper/1308.3397/) (2013)
- [Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Bandits](https://mlsanity.com/paper/1504.06937/) (2015)
- [Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach](https://mlsanity.com/paper/1812.09808/) (2018)
- [Robust Hypothesis Testing Using Wasserstein Uncertainty Sets](https://mlsanity.com/paper/1805.10611/) (2018)
- [Wasserstein Distributionally Robust Optimization and Variation Regularization](https://mlsanity.com/paper/1712.06050/) (2017)
- [Incorporating Unlabeled Data into Distributionally Robust Learning](https://mlsanity.com/paper/1912.07729/) (2019)
- [Wasserstein Distributionally Robust Kalman Filtering](https://mlsanity.com/paper/1809.08830/) (2018)
- [Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso](https://mlsanity.com/paper/1707.08692/) (2017)
- [Distributionally Robust Inverse Covariance Estimation: The Wasserstein Shrinkage Estimator](https://mlsanity.com/paper/1805.07194/) (2018)
- [Data-driven Optimal Cost Selection for Distributionally Robust Optimization](https://mlsanity.com/paper/1705.07152/) (2017)
- [On Distributionally Robust Chance Constrained Programs with Wasserstein Distance](https://mlsanity.com/paper/1806.07418/) (2018)

## Built on by (most cited first, 30 of 44)

- [Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications](https://mlsanity.com/paper/2111.12444/) (2021)
- [Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learning](https://mlsanity.com/paper/2303.02783/) (2023)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [Rank-based Decomposable Losses in Machine Learning: A Survey](https://mlsanity.com/paper/2207.08768/) (2022)
- [The Geometry of Adversarial Training in Binary Classification](https://mlsanity.com/paper/2111.13613/) (2021)
- [Robust LLM Alignment via Distributionally Robust Direct Preference Optimization](https://mlsanity.com/paper/2502.01930/) (2025)
- [AdvST: Revisiting Data Augmentations for Single Domain Generalization](https://mlsanity.com/paper/2312.12720/) (2023)
- [Unifying Distributionally Robust Optimization via Optimal Transport Theory](https://mlsanity.com/paper/2308.05414/) (2023)
- [Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learning](https://mlsanity.com/paper/2301.12593/) (2023)
- [Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage](https://mlsanity.com/paper/2310.18434/) (2023)
- [Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guarantees](https://mlsanity.com/paper/2301.13375/) (2023)
- [On the regularized risk of distributionally robust learning over deep neural networks](https://mlsanity.com/paper/2109.06294/) (2021)
- [Stochastic Optimization for Spectral Risk Measures](https://mlsanity.com/paper/2212.05149/) (2022)
- [Solving a Research Problem in Mathematical Statistics with AI Assistance](https://mlsanity.com/paper/2511.18828/) (2025)
- [On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce it](https://mlsanity.com/paper/2301.03662/) (2023)
- [Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificity](https://mlsanity.com/paper/2301.13565/) (2023)
- [Weak Supervision Performance Evaluation via Partial Identification](https://mlsanity.com/paper/2312.04601/) (2023)
- [textttskwdro: a library for Wasserstein distributionally robust machine learning](https://mlsanity.com/paper/2410.21231/) (2024)
- [Distributionally Robust Optimization via Diffusion Ambiguity Modeling](https://mlsanity.com/paper/2510.22757/) (2025)
- [Distributionally Robust Multiclass Classification and Applications in Deep Image Classifiers](https://mlsanity.com/paper/2109.12772/) (2021)
- [Distributionally Robust Active Learning for Gaussian Process Regression](https://mlsanity.com/paper/2502.16870/) (2025)
- [Distributionally Robust Optimization via Generative Ambiguity Modeling](https://mlsanity.com/paper/2602.08976/) (2026)
- [Distributional Robustness Bounds Generalization Errors](https://mlsanity.com/paper/2212.09962/) (2022)
- [Learning Optimal Classification Trees Robust to Distribution Shifts](https://mlsanity.com/paper/2310.17772/) (2023)
- [Wasserstein Distributionally Robust Shallow Convex Neural Networks](https://mlsanity.com/paper/2407.16800/) (2024)
- [Learning a Single Neuron Robustly to Distributional Shifts and Adversarial Label Noise](https://mlsanity.com/paper/2411.06697/) (2024)
- [Welfare as a Guiding Principle for Machine Learning -- From Compass, to Lens, to Roadmap](https://mlsanity.com/paper/2502.11981/) (2025)
- [Eikonal depth: an optimal control approach to statistical depths](https://mlsanity.com/paper/2201.05274/) (2022)
- [Minimax Statistical Estimation under Wasserstein Contamination](https://mlsanity.com/paper/2308.01853/) (2023)
- [Strong Duality Relations in Nonconvex Risk-Constrained Learning](https://mlsanity.com/paper/2312.01110/) (2023)

## Abstract (arXiv)

This monograph develops a comprehensive statistical learning framework that is robust to (distributional) perturbations in the data using Distributionally Robust Optimization (DRO) under the Wasserstein metric. Beginning with fundamental properties of the Wasserstein metric and the DRO formulation, we explore duality to arrive at tractable formulations and develop finite-sample, as well as asymptotic, performance guarantees. We consider a series of learning problems, including (i) distributionally robust linear regression; (ii) distributionally robust regression with group structure in the predictors; (iii) distributionally robust multi-output regression and multiclass classification, (iv) optimal decision making that combines distributionally robust regression with nearest-neighbor estimation; (v) distributionally robust semi-supervised learning, and (vi) distributionally robust reinforcement learning. A tractable DRO relaxation for each problem is being derived, establishing a connection between robustness and regularization, and obtaining bounds on the prediction and estimation errors of the solution. Beyond theory, we include numerical experiments and case studies using synthetic and real data. The real data experiments are all associated with various health informatics problems, an application area which provided the initial impetus for this work.