mlsanityData to 7 Oct 2026

The geometry of the deep linear network

arXiv 2411.09004 ↗First posted 13 Nov 2024cs.NE math.DS math.PRcited by 10

Not on the map. Why?

Abstract (arXiv)

This article provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning. The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Nov 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 4.

10 papers.

  1. 2025An entropy formula for the Deep Linear Networkcited by 8
  2. 2025Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Regioncited by 5
  3. 2025Regularization Implies balancedness in the deep linear networkcited by 4
  4. 2025The Effect of Depth on the Expressivity of Deep Linear State-Space Modelscited by 1
  5. 2025Geodesics in the Deep Linear Networkcited by 0
  6. 2025On the Convergence of Overparameterized Problems: Inherent Properties of the Compositional Structure of Neural Networkscited by 0
  7. 2026On the implicit regularization of Langevin dynamics with projected noisecited by 0
  8. 2026On the Entropy Formula for Real, Complex, and Quaternionic Deep Linear Networkscited by 0
  9. 2026Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networkscited by 0
  10. 2026Special Lagrangian cones in Deep Learningcited by 0