# The geometry of the deep linear network

arXiv: https://arxiv.org/abs/2411.09004 · HTML: https://mlsanity.com/paper/2411.09004/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2024.

## Built on by (most cited first, 11 of 11)

- [An entropy formula for the Deep Linear Network](https://mlsanity.com/paper/2509.09088/) (2025)
- [Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region](https://mlsanity.com/paper/2509.25351/) (2025)
- [Regularization Implies balancedness in the deep linear network](https://mlsanity.com/paper/2511.01137/) (2025)
- [The Effect of Depth on the Expressivity of Deep Linear State-Space Models](https://mlsanity.com/paper/2506.19296/) (2025)
- [On the implicit regularization of Langevin dynamics with projected noise](https://mlsanity.com/paper/2602.12257/) (2026)
- [Geodesics in the Deep Linear Network](https://mlsanity.com/paper/2510.07324/) (2025)
- [On the Convergence of Overparameterized Problems: Inherent Properties of the Compositional Structure of Neural Networks](https://mlsanity.com/paper/2511.09810/) (2025)
- [Rapid mixing for Gibbs measures in Riemannian manifolds](https://mlsanity.com/paper/2606.13453/) (2026)
- [On the Entropy Formula for Real, Complex, and Quaternionic Deep Linear Networks](https://mlsanity.com/paper/2606.16579/) (2026)
- [Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks](https://mlsanity.com/paper/2608.06597/) (2026)
- [Special Lagrangian cones in Deep Learning](https://mlsanity.com/paper/2609.20159/) (2026)

## Abstract (arXiv)

This article provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning. The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions.