# SGD with Variance Reduction beyond Empirical Risk Minimization

arXiv: https://arxiv.org/abs/1510.04822 · HTML: https://mlsanity.com/paper/1510.04822/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2015.

## Built on by (most cited first, 2 of 2)

- [Do We Need to Penalize Variance of Losses for Learning with Label Noise?](https://mlsanity.com/paper/2201.12739/) (2022)
- [Scalable Cox Regression via Grouped Risk Sets and Sharper LogSumExp Rates](https://mlsanity.com/paper/2609.40120/) (2026)

## Abstract (arXiv)

We introduce a doubly stochastic proximal gradient algorithm for optimizing a finite average of smooth convex functions, whose gradients depend on numerically expensive expectations. Our main motivation is the acceleration of the optimization of the regularized Cox partial-likelihood (the core model used in survival analysis), but our algorithm can be used in different settings as well. The proposed algorithm is doubly stochastic in the sense that gradient steps are done using stochastic gradient descent (SGD) with variance reduction, where the inner expectations are approximated by a Monte-Carlo Markov-Chain (MCMC) algorithm. We derive conditions on the MCMC number of iterations guaranteeing convergence, and obtain a linear rate of convergence under strong convexity and a sublinear rate without this assumption. We illustrate the fact that our algorithm improves the state-of-the-art solver for regularized Cox partial-likelihood on several datasets from survival analysis.