mlsanityData to 14 Oct 2026

An entropy formula for the Deep Linear Network

arXiv 2509.09088 ↗First posted 11 Sep 2025cs.LG math.DG math.DScited by 8

Not on the map. Why?

Abstract (arXiv)

We study the Riemannian geometry of the Deep Linear Network (DLN) as a foundation for a thermodynamic description of the learning process. The main tools are the use of group actions to analyze overparametrization and the use of Riemannian submersion from the space of parameters to the space of observables. The foliation of the balanced manifold in the parameter space by group orbits is used to define and compute a Boltzmann entropy. We also show that the Riemannian geometry on the space of observables defined in [2] is obtained by Riemannian submersion of the balanced manifold. The main technical step is an explicit construction of an orthonormal basis for the tangent space of the balanced manifold using the theory of Jacobi matrices.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

18 of 31 references.

References · 18

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  2. 2017Implicit Regularization in Matrix Factorizationcited by 322
  3. 2018On the Optimization of Deep Networks: Implicit Acceleration by Overparameterizationcited by 284
  4. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  5. 2019Two models of double descent for weak featurescited by 195
  6. 2020Directional convergence and alignment in deep learningcited by 178
  7. 2017No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysiscited by 129
  8. 2022On the Implicit Bias in Deep-Learning Algorithmscited by 86
  9. 2019Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizerscited by 61
  10. 2021Geometry of Linear Convolutional Networkscited by 27
  11. 2021Convergence of gradient descent for learning linear neural networkscited by 22
  12. 2022Infinite-width limit of deep linear neural networkscited by 20
6 more
  1. 2023Function Space and Critical Points of Linear Convolutional Networkscited by 19
  2. 2024The geometry of the deep linear networkcited by 11
  3. 2022Deep Linear Networks for Matrix Completion -- An Infinite Depth Limitcited by 8
  4. 2023Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Losscited by 7
  5. 2023Geometric structure of Deep Learning networks and construction of global mathcal L2 minimizerscited by 6
  6. 2024Implicit Regularization via Spectral Neural Networks and Non-linear Matrix Sensingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Regularization Implies balancedness in the deep linear networkcited by 4
  2. 2026Understanding and inverse design of implicit bias in stochastic learning: a geometric perspectivecited by 3
  3. 2026On the implicit regularization of Langevin dynamics with projected noisecited by 1
  4. 2026Geometric and Spectral Alignment for Deep Neural Network Icited by 1
  5. 2025Geodesics in the Deep Linear Networkcited by 0
  6. 2026On the Entropy Formula for Real, Complex, and Quaternionic Deep Linear Networkscited by 0
  7. 2026Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networkscited by 0
  8. 2026Special Lagrangian cones in Deep Learningcited by 0