# Identifiability of Deep Polynomial Neural Networks

arXiv: https://arxiv.org/abs/2506.17093 · HTML: https://mlsanity.com/paper/2506.17093/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 96 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 30 of 56)

- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://mlsanity.com/paper/2209.04836/) (2022)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Tensor Decomposition for Signal Processing and Machine Learning](https://mlsanity.com/paper/1607.01668/) (2016)
- [Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition](https://mlsanity.com/paper/1412.6553/) (2014)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [On Linear Identifiability of Learned Representations](https://mlsanity.com/paper/2007.00810/) (2020)
- [On the Expressive Power of Deep Learning: A Tensor Analysis](https://mlsanity.com/paper/1509.05009/) (2015)
- [On the Identifiability of Nonlinear ICA: Sparsity and Beyond](https://mlsanity.com/paper/2206.07751/) (2022)
- [Reverse-Engineering Deep ReLU Networks](https://mlsanity.com/paper/1910.00744/) (2019)
- [Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network](https://mlsanity.com/paper/2008.05441/) (2020)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [On the Expressive Power of Deep Polynomial Neural Networks](https://mlsanity.com/paper/1905.12207/) (2019)
- [On the Symmetries of Deep Learning Models and their Internal Representations](https://mlsanity.com/paper/2205.14258/) (2022)
- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [Loss Landscapes of Regularized Linear Autoencoders](https://mlsanity.com/paper/1901.08168/) (2019)
- [Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEs](https://mlsanity.com/paper/2101.08366/) (2021)
- [Pi-nets: Deep Polynomial Neural Networks](https://mlsanity.com/paper/2003.03828/) (2020)
- [Indeterminacy in Generative Models: Characterization and Strong Identifiability](https://mlsanity.com/paper/2206.00801/) (2022)
- [Expressive power of recurrent neural networks](https://mlsanity.com/paper/1711.00811/) (2017)
- [Regularized linear autoencoders recover the principal components, eventually](https://mlsanity.com/paper/2007.06731/) (2020)
- [Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.00136/) (2021)
- [Parameter identifiability of a deep feedforward ReLU neural network](https://mlsanity.com/paper/2112.12982/) (2021)
- [Learning Two-layer Neural Networks with Symmetric Inputs](https://mlsanity.com/paper/1810.06793/) (2018)
- [Polynomial Neural Fields for Subband Decomposition and Manipulation](https://mlsanity.com/paper/2302.04862/) (2023)
- [From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modeling](https://mlsanity.com/paper/2310.11011/) (2023)
- [Augmenting Deep Classifiers with Polynomial Neural Networks](https://mlsanity.com/paper/2104.07916/) (2021)
- [Geometry of Polynomial Neural Networks](https://mlsanity.com/paper/2402.00949/) (2024)

## Built on by (most cited first, 12 of 12)

- [Most ReLU Networks Admit Identifiable Parameters](https://mlsanity.com/paper/2605.03601/) (2026)
- [Robustness Verification of Polynomial Neural Networks](https://mlsanity.com/paper/2602.06105/) (2026)
- [The Symmetries of Three-Layer ReLU Networks](https://mlsanity.com/paper/2605.18319/) (2026)
- [The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networks](https://mlsanity.com/paper/2507.06367/) (2025)
- [Low-Rank Tensor Decompositions for the Theory of Neural Networks](https://mlsanity.com/paper/2508.18408/) (2025)
- [The Geometry of Polynomial Group Convolutional Neural Networks](https://mlsanity.com/paper/2603.29566/) (2026)
- [Algebraic Invariants of Lightning Self-Attention](https://mlsanity.com/paper/2604.15632/) (2026)
- [Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects](https://mlsanity.com/paper/2605.09609/) (2026)
- [Singular Learning and Occam's Razor in Deep Monomial Networks](https://mlsanity.com/paper/2606.28464/) (2026)
- [Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Models](https://mlsanity.com/paper/2608.20183/) (2026)
- [Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networks](https://mlsanity.com/paper/2608.27113/) (2026)
- [On Parameter Symmetries and Conservation Laws in Gradient Flow](https://mlsanity.com/paper/2609.34549/) (2026)

## Abstract (arXiv)

Polynomial Neural Networks (PNNs) possess a rich algebraic and geometric structure. However, their identifiability -- a key property for ensuring interpretability -- remains poorly understood. In this work, we present a comprehensive analysis of the identifiability of deep PNNs, including architectures with and without bias terms. Our results reveal an intricate interplay between activation degrees and layer widths in achieving identifiability. As special cases, we show that architectures with non-increasing layer widths are generically identifiable under mild conditions, while encoder-decoder networks are identifiable when the decoder widths do not grow too rapidly compared to the activation degrees. Our proofs are constructive and center on a connection between deep PNNs and low-rank tensor decompositions, and Kruskal-type uniqueness theorems. We also settle an open conjecture on the dimension of PNN's neurovarieties, and provide new bounds on the activation degrees required for it to reach the expected dimension.