# How regularization affects the geometry of loss functions

arXiv: https://arxiv.org/abs/2307.15744 · HTML: https://mlsanity.com/paper/2307.15744/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 14 references that are in the map.
- First posted on arXiv 28 Jul 2023.

## Builds on (most cited first, 3 of 3)

- [The loss surface of deep linear networks viewed through the algebraic geometry lens](https://mlsanity.com/paper/1810.07716/) (2018)
- [The critical locus of overparameterized neural networks](https://mlsanity.com/paper/2005.04210/) (2020)
- [Lecture Notes: Neural Network Architectures](https://mlsanity.com/paper/2304.05133/) (2023)

## Built on by (most cited first, 1 of 1)

- [When majority rules, minority loses: bias amplification of gradient descent](https://mlsanity.com/paper/2505.13122/) (2025)

## Abstract (arXiv)

What neural networks learn depends fundamentally on the geometry of the underlying loss function. We study how different regularizers affect the geometry of this function. One of the most basic geometric properties of a smooth function is whether it is Morse or not. For nonlinear deep neural networks, the unregularized loss function L is typically not Morse. We consider several different regularizers, including weight decay, and study for which regularizers the regularized function Lε becomes Morse.