# Symmetry & Critical Points

arXiv: https://arxiv.org/abs/2408.14445 · HTML: https://mlsanity.com/paper/2408.14445/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 70 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 14 of 14)

- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond](https://mlsanity.com/paper/1611.07476/) (2016)
- [Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup](https://mlsanity.com/paper/1906.08632/) (2019)
- [On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias](https://mlsanity.com/paper/2205.09072/) (2022)
- [Analytic Characterization of the Hessian in Shallow ReLU Models: A Tale of Symmetry](https://mlsanity.com/paper/2008.01805/) (2020)
- [Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II](https://mlsanity.com/paper/2107.10370/) (2021)
- [On the Principle of Least Symmetry Breaking in Shallow ReLU Models](https://mlsanity.com/paper/1912.11939/) (2019)
- [Symmetry & critical points for a model shallow neural network](https://mlsanity.com/paper/2003.10576/) (2020)
- [Symmetry Breaking in Symmetric Tensor Decomposition](https://mlsanity.com/paper/2103.06234/) (2021)
- [Annihilation of Spurious Minima in Two-Layer ReLU Networks](https://mlsanity.com/paper/2210.06088/) (2022)
- [Should Under-parameterized Student Networks Copy or Average Teacher Weights?](https://mlsanity.com/paper/2311.01644/) (2023)
- [Equivariant bifurcation, quadratic equivariants, and symmetry breaking for the standard representation of Sn](https://mlsanity.com/paper/2107.02422/) (2021)
- [Symmetry & Critical Points for Symmetric Tensor Decomposition Problems](https://mlsanity.com/paper/2306.07886/) (2023)
- [Hidden Minima in Two-Layer ReLU Networks](https://mlsanity.com/paper/2312.16819/) (2023)

## Built on by (most cited first, 2 of 2)

- [Geometry and Optimization of Shallow Polynomial Networks](https://mlsanity.com/paper/2501.06074/) (2025)
- [Eigenvalues of the Hessian in Deep Learning: The Origin of Symmetry and Its Breaking](https://mlsanity.com/paper/2610.09919/) (2026)

## Abstract (arXiv)

Critical points of an invariant function may or may not be symmetric. We prove, however, that if a symmetric critical point exists, those adjacent to it are generically symmetry breaking. This mathematical mechanism is shown to carry important implications for our ability to efficiently minimize invariant nonconvex functions, in particular those associated with neural networks.