# What killed the Convex Booster ?

arXiv: https://arxiv.org/abs/2205.09628 · HTML: https://mlsanity.com/paper/2205.09628/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 80 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 24 of 24)

- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Learning with Symmetric Label Noise: The Importance of Being Unhinged](https://mlsanity.com/paper/1505.07634/) (2015)
- [A Second-Order Approach to Learning with Instance-Dependent Label Noise](https://mlsanity.com/paper/2012.11854/) (2020)
- [Classification with Asymmetric Label Noise: Consistency and Maximal Denoising](https://mlsanity.com/paper/1303.1208/) (2013)
- [Robust Bi-Tempered Logistic Loss Based on Bregman Divergences](https://mlsanity.com/paper/1906.03361/) (2019)
- [Learning with Bounded Instance- and Label-dependent Label Noise](https://mlsanity.com/paper/1709.03768/) (2017)
- [On Symmetric Losses for Learning from Corrupted Labels](https://mlsanity.com/paper/1901.09314/) (2019)
- [Fast rates in statistical and online learning](https://mlsanity.com/paper/1507.02592/) (2015)
- [Calibrated Surrogate Losses for Adversarially Robust Classification](https://mlsanity.com/paper/2005.13748/) (2020)
- [Boosting Simple Learners](https://mlsanity.com/paper/2001.11704/) (2020)
- [Efficient, Noise-Tolerant, and Private Learning via Boosting](https://mlsanity.com/paper/2002.01100/) (2020)
- [Two-temperature logistic regression based on the Tsallis divergence](https://mlsanity.com/paper/1705.07210/) (2017)
- [Being Properly Improper](https://mlsanity.com/paper/2106.09920/) (2021)
- [Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss](https://mlsanity.com/paper/1812.01815/) (2018)
- [Boosting in the Presence of Massart Noise](https://mlsanity.com/paper/2106.07779/) (2021)
- [Double Ramp Loss Based Reject Option Classifier](https://mlsanity.com/paper/1311.6556/) (2013)
- [CCMN: A General Framework for Learning with Class-Conditional Multi-Label Noise](https://mlsanity.com/paper/2105.07338/) (2021)
- [Supervised Learning: No Loss No Cry](https://mlsanity.com/paper/2002.03555/) (2020)
- [Communication Efficient Distributed Agnostic Boosting](https://mlsanity.com/paper/1506.06318/) (2015)
- [On the Error Resistance of Hinge Loss Minimization](https://mlsanity.com/paper/2012.00989/) (2020)
- [Boosting in the presence of outliers: adaptive classification with non-convex loss functions](https://mlsanity.com/paper/1510.01064/) (2015)
- [Sparse Robust Classification via the Kernel Mean](https://mlsanity.com/paper/1506.01520/) (2015)
- [Cost Sensitive Learning in the Presence of Symmetric Label Noise](https://mlsanity.com/paper/1901.02271/) (2019)
- [IPBoost -- Non-Convex Boosting via Integer Programming](https://mlsanity.com/paper/2002.04679/) (2020)

## Built on by (most cited first, 1 of 1)

- [Smoothly Giving up: Robustness for Simple Models](https://mlsanity.com/paper/2302.09114/) (2023)

## Abstract (arXiv)

A landmark negative result of Long and Servedio established a worst-case spectacular failure of a supervised learning trio (loss, algorithm, model) otherwise praised for its high precision machinery. Hundreds of papers followed up on the two suspected culprits: the loss (for being convex) and/or the algorithm (for fitting a classical boosting blueprint). Here, we call to the half-century+ founding theory of losses for class probability estimation (properness), an extension of Long and Servedio's results and a new general boosting algorithm to demonstrate that the real culprit in their specific context was in fact the (linear) model class. We advocate for a more general stanpoint on the problem as we argue that the source of the negative result lies in the dark side of a pervasive -- and otherwise prized -- aspect of ML: textitparameterisation.