# A Survey of Learning Criteria Going Beyond the Usual Risk

arXiv: https://arxiv.org/abs/2110.04996 · HTML: https://mlsanity.com/paper/2110.04996/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 143 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach](https://mlsanity.com/paper/1610.03425/) (2016)
- [Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach](https://mlsanity.com/paper/1506.02188/) (2015)
- [Variance-based regularization with convex objectives](https://mlsanity.com/paper/1610.02581/) (2016)
- [Mean estimation and regression under heavy-tailed distributions--a survey](https://mlsanity.com/paper/1906.04280/) (2019)
- [Tilted Empirical Risk Minimization](https://mlsanity.com/paper/2007.01162/) (2020)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [Fairness risk measures](https://mlsanity.com/paper/1901.08665/) (2019)
- [The Price of Fair PCA: One Extra Dimension](https://mlsanity.com/paper/1811.00103/) (2018)
- [DORO: Distributional and Outlier Robust Optimization](https://mlsanity.com/paper/2106.06142/) (2021)
- [Learning with Average Top-k Loss](https://mlsanity.com/paper/1705.08826/) (2017)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [Adaptive Sampling for Stochastic Risk-Averse Learning](https://mlsanity.com/paper/1910.12511/) (2019)
- [Empirical risk minimization for heavy-tailed losses](https://mlsanity.com/paper/1406.2462/) (2014)
- [Learning Bounds for Risk-sensitive Learning](https://mlsanity.com/paper/2006.08138/) (2020)
- [Minimizing the Maximal Loss: How and Why?](https://mlsanity.com/paper/1602.01690/) (2016)
- [Statistical Learning with Conditional Value at Risk](https://mlsanity.com/paper/2002.05826/) (2020)
- [Risk-Averse Stochastic Convex Bandit](https://mlsanity.com/paper/1810.00737/) (2018)
- [Estimation of Spectral Risk Measures](https://mlsanity.com/paper/1912.10398/) (2019)
- [Spectral risk-based learning using unbounded losses](https://mlsanity.com/paper/2105.04816/) (2021)
- [Learning with risks based on M-location](https://mlsanity.com/paper/2012.02424/) (2020)
- [Robust Generalization despite Distribution Shift via Minimum Discriminating Information](https://mlsanity.com/paper/2106.04443/) (2021)
- [Unifying Lower Bounds on Prediction Dimension of Consistent Convex Surrogates](https://mlsanity.com/paper/2102.08218/) (2021)

## Built on by (most cited first, 4 of 4)

- [Soft ascent-descent as a stable and flexible alternative to flooding](https://mlsanity.com/paper/2310.10006/) (2023)
- [OrderGrad: Optimizing Beyond the Mean with Order-Statistic Policy Gradient Estimation](https://mlsanity.com/paper/2606.06096/) (2026)
- [Robust variance-regularized risk minimization with concomitant scaling](https://mlsanity.com/paper/2301.11584/) (2023)
- [Criterion Collapse and Loss Distribution Control](https://mlsanity.com/paper/2402.09802/) (2024)

## Abstract (arXiv)

Virtually all machine learning tasks are characterized using some form of loss function, and "good performance" is typically stated in terms of a sufficiently small average loss, taken over the random draw of test data. While optimizing for performance on average is intuitive, convenient to analyze in theory, and easy to implement in practice, such a choice brings about trade-offs. In this work, we survey and introduce a wide variety of non-traditional criteria used to design and evaluate machine learning algorithms, place the classical paradigm within the proper historical context, and propose a view of learning problems which emphasizes the question of "what makes for a desirable loss distribution?" in place of tacit use of the expected loss.