mlsanityData to 14 Oct 2026

Identifiability of Deep Polynomial Neural Networks

arXiv 2506.17093 ↗First posted 20 Jun 2025cs.LG cs.AI math.AGcited by 12

Not on the map. Why?

Abstract (arXiv)

Polynomial Neural Networks (PNNs) possess a rich algebraic and geometric structure. However, their identifiability -- a key property for ensuring interpretability -- remains poorly understood. In this work, we present a comprehensive analysis of the identifiability of deep PNNs, including architectures with and without bias terms. Our results reveal an intricate interplay between activation degrees and layer widths in achieving identifiability. As special cases, we show that architectures with non-increasing layer widths are generically identifiable under mild conditions, while encoder-decoder networks are identifiable when the decoder widths do not grow too rapidly compared to the activation degrees. Our proofs are constructive and center on a connection between deep PNNs and low-rank tensor decompositions, and Kruskal-type uniqueness theorems. We also settle an open conjecture on the dimension of PNN's neurovarieties, and provide new bounds on the activation degrees required for it to reach the expected dimension.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 56 of its 96 references that are in the map. First posted on arXiv 20 Jun 2025.

Builds on

56 of 96 references.

References · 56

  1. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  2. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  3. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  4. 2015Tensorizing Neural Networkscited by 250
  5. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 224
  6. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 203
  7. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
  8. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
  9. 2020On Linear Identifiability of Learned Representationscited by 105
  10. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  11. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 85
  12. 2019Reverse-Engineering Deep ReLU Networkscited by 83
38 more
  1. 2020Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Networkcited by 74
  2. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  3. 2019On the Expressive Power of Deep Polynomial Neural Networkscited by 66
  4. 2022On the Symmetries of Deep Learning Models and their Internal Representationscited by 64
  5. 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58
  6. 2019Loss Landscapes of Regularized Linear Autoencoderscited by 46
  7. 2021Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEscited by 43
  8. 2020Pi-nets: Deep Polynomial Neural Networkscited by 39
  9. 2022Indeterminacy in Generative Models: Characterization and Strong Identifiabilitycited by 33
  10. 2017Expressive power of recurrent neural networkscited by 30
  11. 2020Regularized linear autoencoders recover the principal components, eventuallycited by 26
  12. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  13. 2021Parameter identifiability of a deep feedforward ReLU neural networkcited by 24
  14. 2018Learning Two-layer Neural Networks with Symmetric Inputscited by 22
  15. 2023Polynomial Neural Fields for Subband Decomposition and Manipulationcited by 22
  16. 2023From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modelingcited by 22
  17. 2021Augmenting Deep Classifiers with Polynomial Neural Networkscited by 21
  18. 2024Geometry of Polynomial Neural Networkscited by 21
  19. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  20. 2021An Embedding of ReLU Networks and an Analysis of their Identifiabilitycited by 20
  21. 2021On Expressivity and Trainability of Quadratic Networkscited by 19
  22. 2022The Spectral Bias of Polynomial Neural Networkscited by 19
  23. 2020Affine symmetries and neural network identifiabilitycited by 18
  24. 2022Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learningcited by 16
  25. 2024Geometry of Lightning Self-Attention: Identifiability and Dimensioncited by 16
  26. 2021Efficient Algorithms for Learning Depth-2 Neural Networks with General ReLU Activationscited by 15
  27. 2023Expand-and-Cluster: Parameter Recovery of Neural Networkscited by 15
  28. 2018On the Connection Between Learning Two-Layers Neural Networks and Tensor Decompositioncited by 14
  29. 2019Neural network identifiability for a family of sigmoidal nonlinearitiescited by 14
  30. 2023Tensor Decomposition for Model Reduction in Neural Networks: A Reviewcited by 14
  31. 2024Activation degree thresholds and expressiveness of polynomial neural networkscited by 14
  32. 2022Local Identifiability of Deep ReLU Neural Networks: the Theorycited by 12
  33. 2023Geometry-aware training of factorized layers in tensor Tucker formatcited by 12
  34. 2016Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithmscited by 11
  35. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  36. 2025Geometry and Optimization of Shallow Polynomial Networkscited by 10
  37. 2013Uniqueness of Tensor Decompositions with Applications to Polynomial Identifiabilitycited by 9
  38. 2021PolyNet: Polynomial Neural Network for 3D Shape Recognition with PolyShape Representationcited by 9

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.

  1. 2026Most ReLU Networks Admit Identifiable Parameterscited by 6
  2. 2026Robustness Verification of Polynomial Neural Networkscited by 3
  3. 2026The Symmetries of Three-Layer ReLU Networkscited by 3
  4. 2025The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networkscited by 2
  5. 2025Low-Rank Tensor Decompositions for the Theory of Neural Networkscited by 2
  6. 2026The Geometry of Polynomial Group Convolutional Neural Networkscited by 2
  7. 2026Algebraic Invariants of Lightning Self-Attentioncited by 1
  8. 2026Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defectscited by 1
  9. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0
  10. 2026Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Modelscited by 0
  11. 2026Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networkscited by 0
  12. 2026On Parameter Symmetries and Conservation Laws in Gradient Flowcited by 0