# Metric Entropy-Free Sample Complexity Bounds for Sample Average Approximation in Convex Stochastic Programming

arXiv: https://arxiv.org/abs/2401.00664 · HTML: https://mlsanity.com/paper/2401.00664/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 75 references that are in the map.
- First posted on arXiv 1 Jan 2024.

## Builds on (most cited first, 17 of 17)

- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Two models of double descent for weak features](https://mlsanity.com/paper/1903.07571/) (2019)
- [High probability generalization bounds for uniformly stable algorithms with nearly optimal rate](https://mlsanity.com/paper/1902.10710/) (2019)
- [Accelerated Gradient Methods for Nonconvex Nonlinear and Stochastic Programming](https://mlsanity.com/paper/1310.3787/) (2013)
- [Generalization Bounds for Uniformly Stable Algorithms](https://mlsanity.com/paper/1812.09859/) (2018)
- [Stability and Deviation Optimal Risk Bounds with Convergence Rate O(1/n)](https://mlsanity.com/paper/2103.12024/) (2021)
- [Statistical consistency and asymptotic normality for high-dimensional robust M-estimators](https://mlsanity.com/paper/1501.00312/) (2015)
- [Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimization](https://mlsanity.com/paper/1905.11957/) (2019)
- [An extension of McDiarmid's inequality](https://mlsanity.com/paper/1511.05240/) (2015)
- [Sample average approximation with heavier tails I: non-asymptotic bounds with weak assumptions and stochastic constraints](https://mlsanity.com/paper/1705.00822/) (2017)
- [Sample Average Approximations of Strongly Convex Stochastic Programs in Hilbert Spaces](https://mlsanity.com/paper/2104.05114/) (2021)
- [Dimensionality-Dependent Generalization Bounds for k-Dimensional Coding Schemes](https://mlsanity.com/paper/1601.00238/) (2016)
- [Non-asymptotic confidence bounds for the optimal value of a stochastic program](https://mlsanity.com/paper/1601.07592/) (2016)
- [Uses of Sub-sample Estimates to Reduce Errors in Stochastic Optimization Models](https://mlsanity.com/paper/2310.07052/) (2023)
- [Regularized Sample Average Approximation for High-Dimensional Stochastic Optimization Under Low-Rankness](https://mlsanity.com/paper/1904.03453/) (2019)
- [General Feasibility Bounds for Sample Average Approximation via Vapnik-Chervonenkis Dimension](https://mlsanity.com/paper/2103.01324/) (2021)
- [First order asymptotics of the sample average approximation method to solve risk averse stochastic progams](https://mlsanity.com/paper/2107.13863/) (2021)

## Built on by (most cited first, 1 of 1)

- [VC Theory for Inventory Policies](https://mlsanity.com/paper/2404.11509/) (2024)

## Abstract (arXiv)

This paper studies sample average approximation (SAA) in solving convex or strongly convex stochastic programming (SP) problems. In estimating SAA's sample efficiency, the state-of-the-art sample complexity bounds entail metric entropy terms (such as the logarithm of the feasible region's covering number), which often grow polynomially with problem dimensionality. While it has been shown that metric entropy-free complexity rates are attainable under a uniform Lipschitz condition, such an assumption can be overly critical for many important SP problem settings. In response, this paper presents metric entropy-free sample complexity bounds for the SAA under standard SP assumptions -- in the absence of the uniform Lipschitz condition. For a d-dimensional problem, the new results often lead to an O(d)-improvement in the complexity rate compared with the state-of-the-art. From the newly established complexity bounds, an important revelation is that SAA and the canonical stochastic mirror descent (SMD) method, two mainstream solution approaches to SP, entail almost identical rates of sample efficiency, lifting a theoretical discrepancy of SAA from SMD also by a factor of O(d). Furthermore, this paper explores non-Lipschitzian scenarios where SAA maintains provable efficacy but the corresponding results for SMD remain mostly unexplored, indicating the potential of SAA's better applicability in some irregular settings. The results of our numerical experiments align with our theoretical findings.