mlsanityData to 14 Oct 2026

Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry

arXiv 2501.18915 ↗First posted 31 Jan 2025cs.LG math.AGcited by 21

Not on the map. Why?

Abstract (arXiv)

In this position paper, we promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. To this end, we focus on algebraic models, such as neural networks with polynomial activations, whose associated function spaces are semi-algebraic varieties. We outline a dictionary between algebro-geometric invariants of these varieties, such as dimension, degree, and singularities, and fundamental aspects of machine learning, such as sample complexity, expressivity, training dynamics, and implicit bias. Along the way, we review the literature and discuss ideas beyond the algebraic domain. This work lays the foundations of a research direction bridging algebraic geometry and deep learning, that we refer to as neuroalgebraic geometry.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 32 of its 63 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

32 of 63 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  5. 2018On the Optimization of Deep Networks: Implicit Acceleration by Overparameterizationcited by 284
  6. 2023A Survey on Model Compression for Large Language Modelscited by 269
  7. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  8. 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 177
  9. 2022Topological Deep Learning: Going Beyond Graph Datacited by 97
  10. 2018Tropical Geometry of Deep Neural Networkscited by 95
  11. 2020Rational neural networkscited by 75
  12. 2019On the Expressive Power of Deep Polynomial Neural Networkscited by 66
20 more
  1. 2019Gradient Dynamics of Shallow Univariate ReLU Networkscited by 46
  2. 2023Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flowscited by 45
  3. 2023Hidden symmetries of ReLU networkscited by 39
  4. 2021Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sumscited by 37
  5. 2019Pure and Spurious Critical Points: a Geometric Study of Linear Networkscited by 33
  6. 2021Geometry of Linear Convolutional Networkscited by 27
  7. 2024The Real Tropical Geometry of Neural Networkscited by 24
  8. 2023Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networkscited by 23
  9. 2024Geometry of Polynomial Neural Networkscited by 21
  10. 2023Function Space and Critical Points of Linear Convolutional Networkscited by 19
  11. 2013The Euclidean distance degree of an algebraic varietycited by 16
  12. 2024Geometry of Lightning Self-Attention: Identifiability and Dimensioncited by 16
  13. 2021Arbitrary-Depth Universal Approximation Theorems for Operator Neural Networkscited by 12
  14. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  15. 2024Algebraic Complexity and Neurovariety of Linear Convolutional Networkscited by 10
  16. 2025Geometry and Optimization of Shallow Polynomial Networkscited by 10
  17. 2024Decomposition Polyhedra of Piecewise Linear Functionscited by 9
  18. 2023Covering Number of Real Algebraic Varieties and Beyond: Improved Bounds and Applicationscited by 8
  19. 2021Hausdorff approximations and volume of tubes of singular algebraic setscited by 5
  20. 2019Optimal Transport to a Varietycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 14.

21 papers.

Sort:

By area:

  1. 2025Symmetry in Neural Network Parameter Spacescited by 36
  2. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  3. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  4. 2025Orthogonal Low-rank Adaptation in Lie Groups for Continual Learning of Large Language Modelscited by 7
  5. 2026Identifiable Equivariant Networks are Layerwise Equivariantcited by 7
  6. 2025The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networkscited by 2
  7. 2025Toric geometry of ReLU neural networkscited by 2
  8. 2026Rational Neural Networks have Expressivity Advantagescited by 2
  9. 2026The Geometry of Polynomial Group Convolutional Neural Networkscited by 2
  10. 2025Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrizationcited by 1
  11. 2026Algebraic Invariants of Lightning Self-Attentioncited by 1
  12. 2026Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defectscited by 1
9 more
  1. 2026TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Mergingcited by 1
  2. 2026Towards Verifiable AI with Lightweight Cryptographic Proofs of Inferencecited by 0
  3. 2026Stratifying Reinforcement Learning with Signal Temporal Logiccited by 0
  4. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0
  5. 2026Tubular Neighbourhoods of Pfaffian Sets and Applications to Neural Networkscited by 0
  6. 2026Algebraic Signatures for Structural Learning in Probability Tensorscited by 0
  7. 2026An Investigation of the NeurIPS and ICML 2025 Position Trackscited by 0
  8. 2026Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networkscited by 0
  9. 2026Higher Structures in Deep Learningcited by 0