mlsanityData to 7 Oct 2026

Flexible risk design using bi-directional dispersion

arXiv 2203.14434 ↗First posted 28 Mar 2022stat.ML cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Many novel notions of "risk" (e.g., CVaR, tilted risk, DRO risk) have been proposed and studied, but these risks are all at least as sensitive as the mean to loss tails on the upside, and tend to ignore deviations on the downside. We study a complementary new risk class that penalizes loss deviations in a bi-directional manner, while having more flexibility in terms of tail sensitivity than is offered by mean-variance. This class lets us derive high-probability learning guarantees without explicit gradient clipping, and empirical tests using both simulated and real data illustrate a high degree of control over key properties of the test loss distribution incurred by gradient-based learners.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 64 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

20 of 64 references.

References · 20

  1. 2021Convex optimizationcited by 2,479
  2. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  3. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  4. 2017A General and Adaptive Robust Loss Functioncited by 236
  5. 2019A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networkscited by 213
  6. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  7. 2018Stochastic model-based minimization of weakly convex functionscited by 157
  8. 2016Variance-based regularization with convex objectivescited by 153
  9. 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 142
  10. 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
  11. 2020Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clippingcited by 104
  12. 2020Tilted Empirical Risk Minimizationcited by 103
8 more
  1. 2021High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tailscited by 80
  2. 2021DORO: Distributional and Outlier Robust Optimizationcited by 54
  3. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 42
  4. 2019Adaptive Sampling for Stochastic Risk-Averse Learningcited by 39
  5. 2020Learning Bounds for Risk-sensitive Learningcited by 33
  6. 2019Algorithms of Robust Stochastic Optimization Based on Mirror Descent Methodcited by 32
  7. 2021High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noisecited by 18
  8. 2020Learning with risks based on M-locationcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2.

3 papers.

  1. 2023Soft ascent-descent as a stable and flexible alternative to floodingcited by 3
  2. 2024Meta-Learning with Heterogeneous Taskscited by 2
  3. 2024Making Robust Generalizers Less Rigid with Loss Concentrationcited by 0