# Bias-Variance Decompositions for Margin Losses

arXiv: https://arxiv.org/abs/2204.12155 · HTML: https://mlsanity.com/paper/2204.12155/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 33 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 6 of 6)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Learning with Symmetric Label Noise: The Importance of Being Unhinged](https://mlsanity.com/paper/1505.07634/) (2015)
- [Rethinking Bias-Variance Trade-off for Generalization of Neural Networks](https://mlsanity.com/paper/2002.11328/) (2020)
- [Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition](https://mlsanity.com/paper/2011.03321/) (2020)
- [A Modern Take on the Bias-Variance Tradeoff in Neural Networks](https://mlsanity.com/paper/1810.08591/) (2018)
- [Loss factorization, weakly supervised learning and label noise robustness](https://mlsanity.com/paper/1602.02450/) (2016)

## Built on by (most cited first, 2 of 2)

- [A Unified Theory of Diversity in Ensemble Learning](https://mlsanity.com/paper/2301.03962/) (2023)
- [Bias-variance decompositions: the exclusive privilege of Bregman divergences](https://mlsanity.com/paper/2501.18581/) (2025)

## Abstract (arXiv)

We introduce a novel bias-variance decomposition for a range of strictly convex margin losses, including the logistic loss (minimized by the classic LogitBoost algorithm), as well as the squared margin loss and canonical boosting loss. Furthermore, we show that, for all strictly convex margin losses, the expected risk decomposes into the risk of a "central" model and a term quantifying variation in the functional margin with respect to variations in the training data. These decompositions provide a diagnostic tool for practitioners to understand model overfitting/underfitting, and have implications for additive ensemble models -- for example, when our bias-variance decomposition holds, there is a corresponding "ambiguity" decomposition, which can be used to quantify model diversity.