# Data-Driven Sample Average Approximation with Covariate Information

arXiv: https://arxiv.org/abs/2207.13554 · HTML: https://mlsanity.com/paper/2207.13554/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 93 references that are in the map.
- First posted on arXiv 27 Jul 2022.

## Builds on (most cited first, 22 of 22)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [From Predictive to Prescriptive Analytics](https://mlsanity.com/paper/1402.5481/) (2014)
- [Adaptive Huber Regression](https://mlsanity.com/paper/1706.06991/) (2017)
- [Generalization Bounds in the Predict-then-Optimize Framework](https://mlsanity.com/paper/1905.11488/) (2019)
- [JuMP: A Modeling Language for Mathematical Optimization](https://mlsanity.com/paper/1508.01982/) (2015)
- [Fast Rates for Contextual Linear Optimization](https://mlsanity.com/paper/2011.03030/) (2020)
- [Regularized estimation in sparse high-dimensional time series models](https://mlsanity.com/paper/1311.4175/) (2013)
- [Integrated Conditional Estimation-Optimization](https://mlsanity.com/paper/2110.12351/) (2021)
- [Distributionally robust stochastic programs with side information based on trimmings -- Extended version](https://mlsanity.com/paper/2009.10592/) (2020)
- [A Computational Framework for Multivariate Convex Regression and its Variants](https://mlsanity.com/paper/1509.08165/) (2015)
- [Distributionally Robust Local Non-parametric Conditional Estimation](https://mlsanity.com/paper/2010.05373/) (2020)
- [Iteratively reweighted adaptive lasso for conditional heteroscedastic time series with applications to AR-ARCH type processes](https://mlsanity.com/paper/1502.06557/) (2015)
- [From Predictions to Prescriptions in Multistage Optimization Problems](https://mlsanity.com/paper/1904.11637/) (2019)
- [A New Perspective on Robust M-Estimation: Finite Sample Theory and Applications to Dependence-Adjusted Multiple Testing](https://mlsanity.com/paper/1711.05381/) (2017)
- [Distributionally Robust Optimization with Correlated Data from Vector Autoregressive Processes](https://mlsanity.com/paper/1909.03433/) (2019)
- [Heteroscedasticity-aware residuals-based contextual stochastic optimization](https://mlsanity.com/paper/2101.03139/) (2021)
- [Learning Heteroscedastic Models by Convex Programming under Group Sparsity](https://mlsanity.com/paper/1304.4549/) (2013)
- [Nonparametric estimation in a regression model with additive and multiplicative noise](https://mlsanity.com/paper/1906.07695/) (2019)

## Built on by (most cited first, 14 of 14)

- [A Survey of Contextual Optimization Methods for Decision Making under Uncertainty](https://mlsanity.com/paper/2306.10374/) (2023)
- [Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective](https://mlsanity.com/paper/2304.06833/) (2023)
- [Data-driven Piecewise Affine Decision Rules for Stochastic Programming with Covariate Information](https://mlsanity.com/paper/2304.13646/) (2023)
- [A Functional Model Method for Nonconvex Nonsmooth Conditional Stochastic Optimization](https://mlsanity.com/paper/2405.10815/) (2024)
- [Forecasting Outside the Box: Application-Driven Optimal Pointwise Forecasts for Stochastic Optimization](https://mlsanity.com/paper/2411.03520/) (2024)
- [Smart Surrogate Losses for Contextual Stochastic Linear Optimization with Robust Constraints](https://mlsanity.com/paper/2505.22881/) (2025)
- [Decision-Focused Federated Learning Under Heterogeneous Objectives and Constraints](https://mlsanity.com/paper/2604.20031/) (2026)
- ["Over-optimizing" for Normality: Budget-constrained Uncertainty Quantification for Contextual Decision-making](https://mlsanity.com/paper/2503.12747/) (2025)
- [Toward Decision-Oriented Prognostics: An Integrated Estimate-Optimize Framework for Predictive Maintenance](https://mlsanity.com/paper/2506.19698/) (2025)
- [Residuals-based Offline Reinforcement Learning](https://mlsanity.com/paper/2604.01378/) (2026)
- [Fast Convergence of Policy Regret in Learning Stochastic Optimal Control](https://mlsanity.com/paper/2605.26361/) (2026)
- [Learning Optimization Proxies for Sequential Contextual Stochastic Programs: An Order Fulfillment Application](https://mlsanity.com/paper/2606.25362/) (2026)
- [Smooth Learning with Hard Constraints via Legendre-Regularized Policies](https://mlsanity.com/paper/2607.24007/) (2026)
- [Decision-Driven Regularization: A Blended Model for Learning and Optimization](https://mlsanity.com/paper/2608.15124/) (2026)

## Abstract (arXiv)

We study optimization for data-driven decision-making when we have observations of the uncertain parameters within the optimization model together with concurrent observations of covariates. Given a new covariate observation, the goal is to choose a decision that minimizes the expected cost conditioned on this observation. We investigate three data-driven frameworks that integrate a machine learning prediction model within a stochastic programming sample average approximation (SAA) for approximating the solution to this problem. Two of the SAA frameworks are new and use out-of-sample residuals of leave-one-out prediction models for scenario generation. The frameworks we investigate are flexible and accommodate parametric, nonparametric, and semiparametric regression techniques. We derive conditions on the data generation process, the prediction model, and the stochastic program under which solutions of these data-driven SAAs are consistent and asymptotically optimal, and also derive convergence rates and finite sample guarantees. Computational experiments validate our theoretical results, demonstrate the potential advantages of our data-driven formulations over existing approaches (even when the prediction model is misspecified), and illustrate the benefits of our new data-driven formulations in the limited data regime.