mlsanityData to 7 Oct 2026

The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networks

arXiv 2507.06367 ↗First posted 8 Jul 2025cs.LG math.AGcited by 2

Not on the map. Why?

Abstract (arXiv)

We study geometric properties of the gradient flow for learning deep linear convolutional networks. For linear fully connected networks, it has been shown recently that the corresponding gradient flow on parameter space can be written as a Riemannian gradient flow on function space (i.e., on the product of weight matrices) if the initialization satisfies a so-called balancedness condition. We establish that the gradient flow on parameter space for learning linear convolutional networks can be written as a Riemannian gradient flow on function space regardless of the initialization. This result holds for D-dimensional convolutions with D ≥ 2, and for D =1 it holds if all so-called strides of the convolutions are greater than one. The corresponding Riemannian metric depends on the initialization.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 56 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

39 of 56 references.

References · 39

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016Group Equivariant Convolutional Networkscited by 1,154
  3. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  4. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  5. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  6. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  7. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  8. 2018On the Optimization of Deep Networks: Implicit Acceleration by Overparameterizationcited by 280
  9. 2014The Loss Surfaces of Multilayer Networkscited by 278
  10. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  11. 2016Deep Learning without Poor Local Minimacited by 233
  12. 2018Gradient descent aligns the layers of deep linear networkscited by 182
27 more
  1. 2018A Convergence Analysis of Gradient Descent for Deep Linear Neural Networkscited by 180
  2. 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
  3. 2019Dynamics of Deep Neural Networks and Neural Tangent Hierarchycited by 96
  4. 2018The Singular Values of Convolutional Layerscited by 95
  5. 2019Scaling description of generalization with number of parameters in deep learningcited by 83
  6. 2016Understanding Deep Convolutional Networkscited by 76
  7. 2020Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracycited by 70
  8. 2020Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topologycited by 66
  9. 2019On the Expressive Power of Deep Polynomial Neural Networkscited by 65
  10. 2019Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizerscited by 61
  11. 2023Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flowscited by 44
  12. 2023Hidden symmetries of ReLU networkscited by 38
  13. 2016Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regionscited by 33
  14. 2021Geometry of Linear Convolutional Networkscited by 27
  15. 2015Global Optimality in Tensor Factorization, Deep Learning, and Beyondcited by 22
  16. 2021Convergence of gradient descent for learning linear neural networkscited by 22
  17. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  18. 2018The loss surface of deep linear networks viewed through the algebraic geometry lenscited by 20
  19. 2022Functional dimension of feedforward ReLU neural networkscited by 20
  20. 2023Function Space and Critical Points of Linear Convolutional Networkscited by 19
  21. 2024Activation degree thresholds and expressiveness of polynomial neural networkscited by 14
  22. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  23. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  24. 2024Algebraic Complexity and Neurovariety of Linear Convolutional Networkscited by 10
  25. 2025Geometry and Optimization of Shallow Polynomial Networkscited by 10
  26. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  27. 2023Complex Critical Points of Deep Linear Neural Networkscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Intrinsic training dynamics of deep neural networkscited by 3
  2. 2026Convergence of gradient flow for learning convolutional neural networkscited by 0