# Algebraic Complexity and Neurovariety of Linear Convolutional Networks

arXiv: https://arxiv.org/abs/2401.16613 · HTML: https://mlsanity.com/paper/2401.16613/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 13 of 13)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Deep Learning without Poor Local Minima](https://mlsanity.com/paper/1605.07110/) (2016)
- [Deep linear neural networks with arbitrary loss: All local minima are global](https://mlsanity.com/paper/1712.01473/) (2017)
- [Pure and Spurious Critical Points: a Geometric Study of Linear Networks](https://mlsanity.com/paper/1910.01671/) (2019)
- [Geometry of Linear Convolutional Networks](https://mlsanity.com/paper/2108.01538/) (2021)
- [Local minima in training of neural networks](https://mlsanity.com/paper/1611.06310/) (2016)
- [Function Space and Critical Points of Linear Convolutional Networks](https://mlsanity.com/paper/2304.05752/) (2023)
- [The Euclidean distance degree of an algebraic variety](https://mlsanity.com/paper/1309.0049/) (2013)
- [Optimization Theory for ReLU Neural Networks Trained with Normalization Layers](https://mlsanity.com/paper/2006.06878/) (2020)
- [Geometry of Linear Neural Networks: Equivariance and Invariance under Permutation Groups](https://mlsanity.com/paper/2309.13736/) (2023)
- [On the minimal algebraic complexity of the rank-one approximation problem for general inner products](https://mlsanity.com/paper/2309.15105/) (2023)

## Built on by (most cited first, 10 of 10)

- [Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry](https://mlsanity.com/paper/2501.18915/) (2025)
- [On the Geometry and Optimization of Polynomial Convolutional Networks](https://mlsanity.com/paper/2410.00722/) (2024)
- [Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networks](https://mlsanity.com/paper/2505.11846/) (2025)
- [Topology and Geometry of the Learning Space of ReLU Networks: Connectivity and Singularities](https://mlsanity.com/paper/2602.00693/) (2026)
- [Understanding Learning Invariance in Deep Linear Networks](https://mlsanity.com/paper/2506.13714/) (2025)
- [The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networks](https://mlsanity.com/paper/2507.06367/) (2025)
- [The Geometry of Polynomial Group Convolutional Neural Networks](https://mlsanity.com/paper/2603.29566/) (2026)
- [Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrization](https://mlsanity.com/paper/2512.21029/) (2025)
- [Algebraic Invariants of Lightning Self-Attention](https://mlsanity.com/paper/2604.15632/) (2026)
- [Singular Learning and Occam's Razor in Deep Monomial Networks](https://mlsanity.com/paper/2606.28464/) (2026)

## Abstract (arXiv)

In this paper, we study linear convolutional networks with one-dimensional filters and arbitrary strides. The neuromanifold of such a network is a semialgebraic set, represented by a space of polynomials admitting specific factorizations. Introducing a recursive algorithm, we generate polynomial equations whose common zero locus corresponds to the Zariski closure of the corresponding neuromanifold. Furthermore, we explore the algebraic complexity of training these networks employing tools from metric algebraic geometry. Our findings reveal that the number of all complex critical points in the optimization of such a network is equal to the generic Euclidean distance degree of a Segre variety. Notably, this count significantly surpasses the number of critical points encountered in the training of a fully connected linear network with the same number of parameters.