# Achieving First-Order Statistical Improvements in Data-Driven Optimization: From No-Free-Lunch to Amplified Decision Perturbation

arXiv: https://arxiv.org/abs/2608.04312 · HTML: https://mlsanity.com/paper/2608.04312/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 77 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 8 of 8)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Learning from a Biased Sample](https://mlsanity.com/paper/2209.01754/) (2022)
- [Rethinking Distribution Shifts: Empirical Analysis and Modeling for Tabular Data](https://mlsanity.com/paper/2307.05284/) (2023)
- [Dissecting the Impact of Model Misspecification in Data-driven Optimization](https://mlsanity.com/paper/2503.00626/) (2025)
- [A Pareto Dominance Principle for Data-Driven Optimization](https://mlsanity.com/paper/2010.06606/) (2020)
- [Regularization for Adversarial Robust Learning](https://mlsanity.com/paper/2408.09672/) (2024)
- [Distributionally Robust Optimization](https://mlsanity.com/paper/2411.02549/) (2024)
- [Post-estimation Adjustments in Data-driven Decision-making with Applications in Pricing](https://mlsanity.com/paper/2507.20501/) (2025)

## Abstract (arXiv)

Recent proliferation of data-optimization integration has led to a range of methods that aim to improve the statistical performance of data-driven optimization decisions. However, while many of these methods are motivated intuitively from a robustness or regularization perspective, their resulting statistical benefits are often unclear and, even if available, are established on a case-by-case basis. We provide a systematic dissection of data-driven optimization formulations using the view of "directionally perturbed" empirical optimization (EO). Specifically, this umbrella of formulations, which we call "EO+", covers many existing data-driven optimization methods, including regularization, distributionally robust optimization, transfer learning, and analogous methods for contextual optimization. On the one hand, we argue that without additional, correctly specified, side information, any EO+ method can result in at most second-order improvements. This provides a negative conclusion, namely ``no free lunch is possible", on the statistical power of EO+. On the other hand, we show that when leveraging side information that is geometrically effective, achieving first-order improvements is possible by choosing hyperparameters that are significantly larger than what is typically suggested in the literature. Moreover, we construct a principled methodology based on excess risk estimation, via either system knowledge or bootstrap resampling, to maximize the first-order gain. We demonstrate how this gain connects to the control-variate principle, a variance reduction technique in the Monte Carlo simulation literature, which helps explain why geometrically effective side information is necessary.