mlsanityData to 14 Oct 2026

Algebraic Complexity and Neurovariety of Linear Convolutional Networks

arXiv 2401.16613 ↗First posted 29 Jan 2024math.AG cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

In this paper, we study linear convolutional networks with one-dimensional filters and arbitrary strides. The neuromanifold of such a network is a semialgebraic set, represented by a space of polynomials admitting specific factorizations. Introducing a recursive algorithm, we generate polynomial equations whose common zero locus corresponds to the Zariski closure of the corresponding neuromanifold. Furthermore, we explore the algebraic complexity of training these networks employing tools from metric algebraic geometry. Our findings reveal that the number of all complex critical points in the optimization of such a network is equal to the generic Euclidean distance degree of a Segre variety. Notably, this count significantly surpasses the number of critical points encountered in the training of a fully connected linear network with the same number of parameters.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 13 of its 25 references that are in the map. First posted on arXiv 29 Jan 2024.

Builds on

13 of 25 references.

References · 13

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  3. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  4. 2016Deep Learning without Poor Local Minimacited by 234
  5. 2017Deep linear neural networks with arbitrary loss: All local minima are globalcited by 65
  6. 2019Pure and Spurious Critical Points: a Geometric Study of Linear Networkscited by 33
  7. 2021Geometry of Linear Convolutional Networkscited by 27
  8. 2016Local minima in training of neural networkscited by 19
  9. 2023Function Space and Critical Points of Linear Convolutional Networkscited by 19
  10. 2013The Euclidean distance degree of an algebraic varietycited by 16
  11. 2020Optimization Theory for ReLU Neural Networks Trained with Normalization Layerscited by 15
  12. 2023Geometry of Linear Neural Networks: Equivariance and Invariance under Permutation Groupscited by 5
1 more
  1. 2023On the minimal algebraic complexity of the rank-one approximation problem for general inner productscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 4.

10 papers.

  1. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  2. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  3. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  4. 2026Topology and Geometry of the Learning Space of ReLU Networks: Connectivity and Singularitiescited by 6
  5. 2025Understanding Learning Invariance in Deep Linear Networkscited by 3
  6. 2025The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networkscited by 2
  7. 2026The Geometry of Polynomial Group Convolutional Neural Networkscited by 2
  8. 2025Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrizationcited by 1
  9. 2026Algebraic Invariants of Lightning Self-Attentioncited by 1
  10. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0