mlsanityData to 7 Oct 2026

A Survey of Learning Criteria Going Beyond the Usual Risk

arXiv 2110.04996 ↗First posted 11 Oct 2021stat.ML cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Virtually all machine learning tasks are characterized using some form of loss function, and "good performance" is typically stated in terms of a sufficiently small average loss, taken over the random draw of test data. While optimizing for performance on average is intuitive, convenient to analyze in theory, and easy to implement in practice, such a choice brings about trade-offs. In this work, we survey and introduce a wide variety of non-traditional criteria used to design and evaluate machine learning algorithms, place the classical paradigm within the proper historical context, and propose a view of learning problems which emphasizes the question of "what makes for a desirable loss distribution?" in place of tacit use of the expected loss.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 27 of its 143 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

27 of 143 references.

References · 27

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  2. 2020Long-tail learning via logit adjustmentcited by 605
  3. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  4. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  5. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  6. 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 182
  7. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  8. 2016Variance-based regularization with convex objectivescited by 153
  9. 2019Mean estimation and regression under heavy-tailed distributions--a surveycited by 133
  10. 2020Tilted Empirical Risk Minimizationcited by 103
  11. 2021The Bayesian Learning Rulecited by 95
  12. 2019Fairness risk measurescited by 85
15 more
  1. 2018The Price of Fair PCA: One Extra Dimensioncited by 67
  2. 2021DORO: Distributional and Outlier Robust Optimizationcited by 54
  3. 2017Learning with Average Top-k Losscited by 52
  4. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 42
  5. 2019Adaptive Sampling for Stochastic Risk-Averse Learningcited by 39
  6. 2014Empirical risk minimization for heavy-tailed lossescited by 37
  7. 2020Learning Bounds for Risk-sensitive Learningcited by 33
  8. 2016Minimizing the Maximal Loss: How and Why?cited by 30
  9. 2020Statistical Learning with Conditional Value at Riskcited by 19
  10. 2018Risk-Averse Stochastic Convex Banditcited by 13
  11. 2019Estimation of Spectral Risk Measurescited by 9
  12. 2021Spectral risk-based learning using unbounded lossescited by 8
  13. 2020Learning with risks based on M-locationcited by 6
  14. 2021Robust Generalization despite Distribution Shift via Minimum Discriminating Informationcited by 6
  15. 2021Unifying Lower Bounds on Prediction Dimension of Consistent Convex Surrogatescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2023Soft ascent-descent as a stable and flexible alternative to floodingcited by 3
  2. 2026OrderGrad: Optimizing Beyond the Mean with Order-Statistic Policy Gradient Estimationcited by 1
  3. 2023Robust variance-regularized risk minimization with concomitant scalingcited by 0
  4. 2024Criterion Collapse and Loss Distribution Controlcited by 0