# Neural network identifiability for a family of sigmoidal nonlinearities

arXiv: https://arxiv.org/abs/1906.06994 · HTML: https://mlsanity.com/paper/1906.06994/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2019.

## Built on by (most cited first, 14 of 14)

- [Identifiability of Deep Polynomial Neural Networks](https://mlsanity.com/paper/2506.17093/) (2025)
- [Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networks](https://mlsanity.com/paper/2505.11846/) (2025)
- [Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples](https://mlsanity.com/paper/2101.07150/) (2021)
- [Identifiable Equivariant Networks are Layerwise Equivariant](https://mlsanity.com/paper/2601.21645/) (2026)
- [Functional Equivalence and Path Connectivity of Reducible Hyperbolic Tangent Networks](https://mlsanity.com/paper/2305.05089/) (2023)
- [Learning ReLU networks to high uniform accuracy is intractable](https://mlsanity.com/paper/2205.13531/) (2022)
- [Metric Entropy Limits on Recurrent Neural Network Learning of Linear Dynamical Systems](https://mlsanity.com/paper/2105.02556/) (2021)
- [On the space of coefficients of a Feed Forward Neural Network](https://mlsanity.com/paper/2109.03362/) (2021)
- [Conservation Laws from Data Symmetry in Neural Networks](https://mlsanity.com/paper/2606.10913/) (2026)
- [Neural Networks as Linear Regression: An Introduction for Statisticians](https://mlsanity.com/paper/2606.23601/) (2026)
- [Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks](https://mlsanity.com/paper/2609.38998/) (2026)
- [Fine-Grained System Identification of Nonlinear Neural Circuits](https://mlsanity.com/paper/2106.05400/) (2021)
- [Proximity to Losslessly Compressible Parameters](https://mlsanity.com/paper/2306.02834/) (2023)
- [Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networks](https://mlsanity.com/paper/2608.27113/) (2026)

## Abstract (arXiv)

This paper addresses the following question of neural network identifiability: Does the input-output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? Existing literature on the subject Sussman 1992, Albertini, Sontag et al. 1993, Fefferman 1994 suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided the networks under consideration satisfy certain "genericity conditions". The results in Sussman 1992 and Albertini, Sontag et al. 1993 apply to networks with a single hidden layer and in Fefferman 1994 the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.