# All your loss are belong to Bayes

arXiv: https://arxiv.org/abs/2006.04633 · HTML: https://mlsanity.com/paper/2006.04633/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2020.

## Built on by (most cited first, 3 of 3)

- [Being Properly Improper](https://mlsanity.com/paper/2106.09920/) (2021)
- [LegendreTron: Uprising Proper Multiclass Loss Learning](https://mlsanity.com/paper/2301.11695/) (2023)
- [DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory (and its Loss' Convexity is Dispensable)](https://mlsanity.com/paper/2507.07855/) (2025)

## Abstract (arXiv)

Loss functions are a cornerstone of machine learning and the starting point of most algorithms. Statistics and Bayesian decision theory have contributed, via properness, to elicit over the past decades a wide set of admissible losses in supervised learning, to which most popular choices belong (logistic, square, Matsushita, etc.). Rather than making a potentially biased ad hoc choice of the loss, there has recently been a boost in efforts to fit the loss to the domain at hand while training the model itself. The key approaches fit a canonical link, a function which monotonically relates the closed unit interval to R and can provide a proper loss via integration. In this paper, we rely on a broader view of proper composite losses and a recent construct from information geometry, source functions, whose fitting alleviates constraints faced by canonical links. We introduce a trick on squared Gaussian Processes to obtain a random process whose paths are compliant source functions with many desirable properties in the context of link estimation. Experimental results demonstrate substantial improvements over the state of the art.