# Learning Effective Loss Functions Efficiently

arXiv: https://arxiv.org/abs/1907.00103 · HTML: https://mlsanity.com/paper/1907.00103/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2019.

## Built on by (most cited first, 2 of 2)

- [LegendreTron: Uprising Proper Multiclass Loss Learning](https://mlsanity.com/paper/2301.11695/) (2023)
- [Can we infer the presence of Differential Privacy in Deep Learning models' weights? Towards more secure Deep Learning](https://mlsanity.com/paper/2311.11717/) (2023)

## Abstract (arXiv)

We consider the problem of learning a loss function which, when minimized over a training dataset, yields a model that approximately minimizes a validation error metric. Though learning an optimal loss function is NP-hard, we present an anytime algorithm that is asymptotically optimal in the worst case, and is provably efficient in an idealized "easy" case. Experimentally, we show that this algorithm can be used to tune loss function hyperparameters orders of magnitude faster than state-of-the-art alternatives. We also show that our algorithm can be used to learn novel and effective loss functions on-the-fly during training.