mlsanityData to 7 Oct 2026

Geometry of Polynomial Neural Networks

arXiv 2402.00949 ↗First posted 1 Feb 2024math.AG cs.LG stat.MLcited by 21

Not on the map. Why?

Abstract (arXiv)

We study the expressivity and learning process for polynomial neural networks (PNNs) with monomial activation functions. The weights of the network parametrize the neuromanifold. In this paper, we study certain neuromanifolds using tools from algebraic geometry: we give explicit descriptions as semialgebraic sets and characterize their Zariski closures, called neurovarieties. We study their dimension and associate an algebraic degree, the learning degree, to the neurovariety. The dimension serves as a geometric measure for the expressivity of the network, the learning degree is a measure for the complexity of training the network and provides upper bounds on the number of learnable functions. These theoretical results are accompanied with experiments.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 16 of its 41 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

16 of 41 references.

References · 16

  1. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  2. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  3. 2018A Convergence Analysis of Gradient Descent for Deep Linear Neural Networkscited by 180
  4. 2020Better Theory for SGD in the Nonconvex Worldcited by 156
  5. 2018On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizescited by 155
  6. 2016Gradient Descent Learns Linear Dynamical Systemscited by 78
  7. 2020Rational neural networkscited by 75
  8. 2019On the Expressive Power of Deep Polynomial Neural Networkscited by 65
  9. 2021Leveraging Non-uniformity in First-order Non-convex Optimizationcited by 44
  10. 2020Pi-nets: Deep Polynomial Neural Networkscited by 39
  11. 2019Pure and Spurious Critical Points: a Geometric Study of Linear Networkscited by 33
  12. 2022Functional dimension of feedforward ReLU neural networkscited by 20
4 more
  1. 2021PolyNet: Polynomial Neural Network for 3D Shape Recognition with PolyShape Representationcited by 9
  2. 2023Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Losscited by 7
  3. 2019Exponential Convergence for Distributed Smooth Optimization Under the Restricted Secant Inequality Conditioncited by 6
  4. 2022Gradient Descent Is Optimal Under Lower Restricted Secant Inequality And Upper Error Boundcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 10; 2026: 7.

21 papers.

Sort:

  1. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  2. 2024Geometry of Lightning Self-Attention: Identifiability and Dimensioncited by 16
  3. 2024Activation degree thresholds and expressiveness of polynomial neural networkscited by 14
  4. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  5. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  6. 2025Geometry and Optimization of Shallow Polynomial Networkscited by 10
  7. 2024Polynomial Composition Activations: Unleashing the Dynamics of Large Language Modelscited by 9
  8. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  9. 2025The Alexander-Hirschowitz theorem for neurovarietiescited by 6
  10. 2026Robustness Verification of Polynomial Neural Networkscited by 3
  11. 2025Toric geometry of ReLU neural networkscited by 2
  12. 2025Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrizationcited by 1
9 more
  1. 2026Algebraic Invariants of Lightning Self-Attentioncited by 1
  2. 2026Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defectscited by 1
  3. 2025Polynomial, trigonometric, and tropical activationscited by 0
  4. 2025Geometric Properties and Graph-Based Optimization of Neural Networks: Addressing Non-Linearity, Dimensionality, and Scalabilitycited by 0
  5. 2025SUPN: Shallow Universal Polynomial Networkscited by 0
  6. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0
  7. 2026Algebraic Signatures for Structural Learning in Probability Tensorscited by 0
  8. 2026Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Modelscited by 0
  9. 2026Algebraic Expressivity Certificates for Shallow Polynomial Neural Networkscited by 0