# A data-driven approach to beating SAA out-of-sample

arXiv: https://arxiv.org/abs/2105.12342 · HTML: https://mlsanity.com/paper/2105.12342/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 10 of 10)

- [Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach](https://mlsanity.com/paper/1610.03425/) (2016)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Variance-based regularization with convex objectives](https://mlsanity.com/paper/1610.02581/) (2016)
- [Recovering Best Statistical Guarantees via the Empirical Divergence-based Distributionally Robust Optimization](https://mlsanity.com/paper/1605.09349/) (2016)
- [Likelihood Robust Optimization for Data-driven Problems](https://mlsanity.com/paper/1307.6279/) (2013)
- [Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation](https://mlsanity.com/paper/1910.10583/) (2019)
- [The Empirical Likelihood Approach to Quantifying Uncertainty in Sample Average Approximation](https://mlsanity.com/paper/1604.02573/) (2016)
- [Optimistic Distributionally Robust Policy Optimization](https://mlsanity.com/paper/2006.07815/) (2020)
- [Optimistic Robust Optimization With Applications To Machine Learning](https://mlsanity.com/paper/1711.07511/) (2017)
- [Worst-case sensitivity](https://mlsanity.com/paper/2010.10794/) (2020)

## Built on by (most cited first, 3 of 3)

- [Rockafellian Relaxation and Stochastic Optimization under Perturbations](https://mlsanity.com/paper/2204.04762/) (2022)
- [On Tractability, Complexity, and Mixed-Integer Convex Programming Representability of Distributionally Favorable Optimization](https://mlsanity.com/paper/2401.17899/) (2024)
- [An Optimistic-Robust Approach for Dynamic Positioning of Omnichannel Inventories](https://mlsanity.com/paper/2310.12183/) (2023)

## Abstract (arXiv)

While solutions of Distributionally Robust Optimization (DRO) problems can sometimes have a higher out-of-sample expected reward than the Sample Average Approximation (SAA), there is no guarantee. In this paper, we introduce a class of Distributionally Optimistic Optimization (DOO) models, and show that it is always possible to ``beat" SAA out-of-sample if we consider not just worst-case (DRO) models but also best-case (DOO) ones. We also show, however, that this comes at a cost: Optimistic solutions are more sensitive to model error than either worst-case or SAA optimizers, and hence are less robust and calibrating the worst- or best-case model to outperform SAA may be difficult when data is limited.