# Criterion Collapse and Loss Distribution Control

arXiv: https://arxiv.org/abs/2402.09802 · HTML: https://mlsanity.com/paper/2402.09802/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 17 of 17)

- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Tilted Empirical Risk Minimization](https://mlsanity.com/paper/2007.01162/) (2020)
- [Do We Need Zero Training Loss After Achieving Zero Training Error?](https://mlsanity.com/paper/2002.08709/) (2020)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [Adaptive Sampling for Stochastic Risk-Averse Learning](https://mlsanity.com/paper/1910.12511/) (2019)
- [Learning Bounds for Risk-sensitive Learning](https://mlsanity.com/paper/2006.08138/) (2020)
- [Rank-based Decomposable Losses in Machine Learning: A Survey](https://mlsanity.com/paper/2207.08768/) (2022)
- [Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias](https://mlsanity.com/paper/2210.02720/) (2022)
- [Boosted CVaR Classification](https://mlsanity.com/paper/2110.13948/) (2021)
- [What's documented in AI? Systematic Analysis of 32K AI Model Cards](https://mlsanity.com/paper/2402.05160/) (2024)
- [Learning with risks based on M-location](https://mlsanity.com/paper/2012.02424/) (2020)
- [A Survey of Learning Criteria Going Beyond the Usual Risk](https://mlsanity.com/paper/2110.04996/) (2021)
- [Soft ascent-descent as a stable and flexible alternative to flooding](https://mlsanity.com/paper/2310.10006/) (2023)
- [Classification using margin pursuit](https://mlsanity.com/paper/1810.04863/) (2018)
- [Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity](https://mlsanity.com/paper/2208.03066/) (2022)

## Abstract (arXiv)

In this work, we consider the notion of "criterion collapse," in which optimization of one metric implies optimality in another, with a particular focus on conditions for collapse into error probability minimizers under a wide variety of learning criteria, ranging from DRO and OCE risks (CVaR, tilted ERM) to non-monotonic criteria underlying recent ascent-descent algorithms explored in the literature (Flooding, SoftAD). We show how collapse in the context of losses with a Bernoulli distribution goes far beyond existing results for CVaR and DRO, then expand our scope to include surrogate losses, showing conditions where monotonic criteria such as tilted ERM cannot avoid collapse, whereas non-monotonic alternatives can.