mlsanityData to 7 Oct 2026

A Model-Based Method for Minimizing CVaR and Beyond

arXiv 2305.17498 ↗First posted 27 May 2023math.OC cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We develop a variant of the stochastic prox-linear method for minimizing the Conditional Value-at-Risk (CVaR) objective. CVaR is a risk measure focused on minimizing worst-case performance, defined as the average of the top quantile of the losses. In machine learning, such a risk measure is useful to train more robust models. Although the stochastic subgradient method (SGM) is a natural choice for minimizing the CVaR objective, we show that our stochastic prox-linear (SPL+) algorithm can better exploit the structure of the objective, while still providing a convenient closed form update. Our SPL+ method also adapts to the scaling of the loss function, which allows for easier tuning. We then specialize a general convergence theorem for SPL+ to our setting, and show that it allows for a wider selection of step sizes compared to SGM. We support this theoretical finding experimentally.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 16 of its 40 references that are in the map. First posted on arXiv 27 May 2023.

Builds on

16 of 40 references.

References · 16

  1. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  2. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  3. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  4. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  5. 2018Stochastic model-based minimization of weakly convex functionscited by 157
  6. 2014Algorithms for CVaR Optimization in MDPscited by 98
  7. 2019Fairness risk measurescited by 85
  8. 2018Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivitycited by 61
  9. 2017Stochastic Methods for Composite and Weakly Convex Optimization Problemscited by 48
  10. 2019Adaptive Sampling for Stochastic Risk-Averse Learningcited by 39
  11. 2019The importance of better models in stochastic optimizationcited by 32
  12. 2016Minimizing the Maximal Loss: How and Why?cited by 30
4 more
  1. 2020Statistical Learning with Conditional Value at Riskcited by 19
  2. 2022Cutting Some Slack for SGD with Adaptive Polyak Stepsizescited by 18
  3. 2018Risk-Averse Stochastic Convex Banditcited by 13
  4. 2022Superquantiles at Work: Machine Learning Applications and Efficient Subgradient Computationcited by 10

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1; 2025: 1.

5 papers.

  1. 2025An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variantscited by 25
  2. 2023Function Value Learning: Adaptive Learning Rates Based on the Polyak Stepsize and Function Splitting in ERMcited by 8
  3. 2023A Near-Optimal Single-Loop Stochastic Algorithm for Convex Finite-Sum Coupled Compositional Optimizationcited by 3
  4. 2023Multi-Group Fairness Evaluation via Conditional Value-at-Risk Testingcited by 3
  5. 2024On Calibration in Multi-Distribution Learningcited by 1