mlsanityData to 14 Oct 2026

On the Geometry and Optimization of Polynomial Convolutional Networks

arXiv 2410.00722 ↗First posted 1 Oct 2024cs.LG math.AGcited by 11

Not on the map. Why?

Abstract (arXiv)

We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map - typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

18 of 33 references.

References · 18

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2022A ConvNet for the 2020scited by 3,579
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  6. 2020Rational neural networkscited by 75
  7. 2019On the Expressive Power of Deep Polynomial Neural Networkscited by 66
  8. 2017Neural networks and rational functionscited by 40
  9. 2021Geometry of Linear Convolutional Networkscited by 27
  10. 2023Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networkscited by 23
  11. 2024Geometry of Polynomial Neural Networkscited by 21
  12. 2023Function Space and Critical Points of Linear Convolutional Networkscited by 19
6 more
  1. 2013The Euclidean distance degree of an algebraic varietycited by 16
  2. 2024Geometry of Lightning Self-Attention: Identifiability and Dimensioncited by 16
  3. 2019PowerNet: Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Unitscited by 12
  4. 2024Algebraic Complexity and Neurovariety of Linear Convolutional Networkscited by 10
  5. 2021Hausdorff approximations and volume of tubes of singular algebraic setscited by 5
  6. 2023On the minimal algebraic complexity of the rank-one approximation problem for general inner productscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

  1. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  2. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  3. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  4. 2026Identifiable Equivariant Networks are Layerwise Equivariantcited by 7
  5. 2025The Alexander-Hirschowitz theorem for neurovarietiescited by 6
  6. 2025The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networkscited by 2
  7. 2026The Geometry of Polynomial Group Convolutional Neural Networkscited by 2
  8. 2025Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrizationcited by 1
  9. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0
  10. 2026Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networkscited by 0
  11. 2026Grassmann--Pl"ucker Parametrization of Convolutional Filter Subspaces: Regularity and Closed Embeddingscited by 0