# On the Impossibility of Statistically Improving Empirical Optimization: A Second-Order Stochastic Dominance Perspective

arXiv: https://arxiv.org/abs/2105.13419 · HTML: https://mlsanity.com/paper/2105.13419/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2021.

## Built on by (most cited first, 6 of 6)

- [Statistical Analysis of Wasserstein Distributionally Robust Estimators](https://mlsanity.com/paper/2108.02120/) (2021)
- [Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective](https://mlsanity.com/paper/2304.06833/) (2023)
- [Optimizer's Information Criterion: Dissecting and Correcting Bias in Data-Driven Optimization](https://mlsanity.com/paper/2306.10081/) (2023)
- [The Bias-Variance Tradeoff in Data-Driven Optimization: A Local Misspecification Perspective](https://mlsanity.com/paper/2510.18215/) (2025)
- [Wasserstein Distributionally Robust Regret Optimization](https://mlsanity.com/paper/2504.10796/) (2025)
- [Robust Output Analysis with Monte-Carlo Methodology](https://mlsanity.com/paper/2207.13612/) (2022)

## Abstract (arXiv)

When the underlying probability distribution in a stochastic optimization is observed only through data, various data-driven formulations have been studied to obtain approximate optimal solutions. We show that no such formulations can, in a sense, theoretically improve the statistical quality of the solution obtained from empirical optimization. We argue this by proving that the first-order behavior of the optimality gap against the oracle best solution, which includes both the bias and variance, for any data-driven solution is second-order stochastically dominated by empirical optimization, as long as suitable smoothness holds with respect to the underlying distribution. We demonstrate this impossibility of improvement in a range of examples including regularized optimization, distributionally robust optimization, parametric optimization and Bayesian generalizations. We also discuss the connections of our results to semiparametric statistical inference and other perspectives in the data-driven optimization literature.