# Boosted CVaR Classification

arXiv: https://arxiv.org/abs/2110.13948 · HTML: https://mlsanity.com/paper/2110.13948/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [A Rewriting System for Convex Optimization Problems](https://mlsanity.com/paper/1709.04494/) (2017)
- [DORO: Distributional and Outlier Robust Optimization](https://mlsanity.com/paper/2106.06142/) (2021)
- [AdaFair: Cumulative Fairness Adaptive Boosting](https://mlsanity.com/paper/1909.08982/) (2019)
- [Class-Weighted Classification: Trade-offs and Robust Approaches](https://mlsanity.com/paper/2005.12914/) (2020)
- [Modeling the Second Player in Distributionally Robust Optimization](https://mlsanity.com/paper/2103.10282/) (2021)

## Built on by (most cited first, 12 of 12)

- [UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixup](https://mlsanity.com/paper/2209.08928/) (2022)
- [Understanding Why Generalized Reweighting Does Not Improve Over ERM](https://mlsanity.com/paper/2201.12293/) (2022)
- [Distributionally Robust Optimization via Ball Oracle Acceleration](https://mlsanity.com/paper/2203.13225/) (2022)
- [Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts](https://mlsanity.com/paper/2302.02931/) (2023)
- [Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization](https://mlsanity.com/paper/2306.09222/) (2023)
- [A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysis](https://mlsanity.com/paper/2210.04491/) (2022)
- [Reweighted Mixup for Subpopulation Shift](https://mlsanity.com/paper/2304.04148/) (2023)
- [Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective](https://mlsanity.com/paper/2401.14343/) (2024)
- [Responsible AI (RAI) Games and Ensembles](https://mlsanity.com/paper/2310.18832/) (2023)
- [Strong Duality Relations in Nonconvex Risk-Constrained Learning](https://mlsanity.com/paper/2312.01110/) (2023)
- [Robust Long-Tailed Learning via Label-Aware Bounded CVaR](https://mlsanity.com/paper/2308.15405/) (2023)
- [Criterion Collapse and Loss Distribution Control](https://mlsanity.com/paper/2402.09802/) (2024)

## Abstract (arXiv)

Many modern machine learning tasks require models with high tail performance, i.e. high performance over the worst-off samples in the dataset. This problem has been widely studied in fields such as algorithmic fairness, class imbalance, and risk-sensitive decision making. A popular approach to maximize the model's tail performance is to minimize the CVaR (Conditional Value at Risk) loss, which computes the average risk over the tails of the loss. However, for classification tasks where models are evaluated by the zero-one loss, we show that if the classifiers are deterministic, then the minimizer of the average zero-one loss also minimizes the CVaR zero-one loss, suggesting that CVaR loss minimization is not helpful without additional assumptions. We circumvent this negative result by minimizing the CVaR loss over randomized classifiers, for which the minimizers of the average zero-one loss and the CVaR zero-one loss are no longer the same, so minimizing the latter can lead to better tail performance. To learn such randomized classifiers, we propose the Boosted CVaR Classification framework which is motivated by a direct relationship between CVaR and a classical boosting algorithm called LPBoost. Based on this framework, we design an algorithm called α-AdaLPBoost. We empirically evaluate our proposed algorithm on four benchmark datasets and show that it achieves higher tail performance than deterministic model training methods.