# Stochastic Optimization for Spectral Risk Measures

arXiv: https://arxiv.org/abs/2212.05149 · HTML: https://mlsanity.com/paper/2212.05149/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 72 references that are in the map.
- First posted on arXiv 10 Dec 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Large-Scale Methods for Distributionally Robust Optimization](https://mlsanity.com/paper/2010.05893/) (2020)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Variance-based regularization with convex objectives](https://mlsanity.com/paper/1610.02581/) (2016)
- [Tilted Empirical Risk Minimization](https://mlsanity.com/paper/2007.01162/) (2020)
- [The iWildCam 2020 Competition Dataset](https://mlsanity.com/paper/2004.10340/) (2020)
- [Fairness risk measures](https://mlsanity.com/paper/1901.08665/) (2019)
- [Learning with Average Top-k Loss](https://mlsanity.com/paper/1705.08826/) (2017)
- [Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization](https://mlsanity.com/paper/1907.04371/) (2019)
- [Distributionally Robust Learning](https://mlsanity.com/paper/2108.08993/) (2021)
- [Variance Reduced Stochastic Gradient Descent with Neighbors](https://mlsanity.com/paper/1506.03662/) (2015)
- [Adaptive Sampling for Stochastic Risk-Averse Learning](https://mlsanity.com/paper/1910.12511/) (2019)
- [Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning](https://mlsanity.com/paper/1402.4419/) (2014)
- [Learning Bounds for Risk-sensitive Learning](https://mlsanity.com/paper/2006.08138/) (2020)
- [A Wasserstein distance approach for concentration of empirical risk estimates](https://mlsanity.com/paper/1902.10709/) (2019)
- [Superquantiles at Work: Machine Learning Applications and Efficient Subgradient Computation](https://mlsanity.com/paper/2201.00508/) (2022)
- [Estimation of Spectral Risk Measures](https://mlsanity.com/paper/1912.10398/) (2019)
- [Spectral risk-based learning using unbounded losses](https://mlsanity.com/paper/2105.04816/) (2021)
- [First-order Optimization for Superquantile-based Supervised Learning](https://mlsanity.com/paper/2009.14575/) (2020)

## Built on by (most cited first, 7 of 7)

- [Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond](https://mlsanity.com/paper/2403.03562/) (2024)
- [OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework](https://mlsanity.com/paper/2606.08574/) (2026)
- [On Calibration in Multi-Distribution Learning](https://mlsanity.com/paper/2412.14142/) (2024)
- [When fairness metrics fail: A utility-based perspective on varepsilon-fairness](https://mlsanity.com/paper/2405.09360/) (2024)
- [SOREL: A Stochastic Algorithm for Spectral Risks Minimization](https://mlsanity.com/paper/2407.14618/) (2024)
- [Optimizing Shortfall Risk Metric for Learning Regression Models](https://mlsanity.com/paper/2505.17777/) (2025)
- [Convergence of empirical subgradients for optimal transport-based objectives](https://mlsanity.com/paper/2605.28134/) (2026)

## Abstract (arXiv)

Spectral risk objectives - also called L-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop stochastic algorithms to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging are hindered by bias and that our approach outperforms them.