mlsanityData to 14 Oct 2026

Parameter identifiability of a deep feedforward ReLU neural network

arXiv 2112.12982 ↗First posted 24 Dec 2021math.ST stat.ML stat.THcited by 24

Not on the map. Why?

Abstract (arXiv)

The possibility for one to recover the parameters-weights and biases-of a neural network thanks to the knowledge of its function on a subset of the input space can be, depending on the situation, a curse or a blessing. On one hand, recovering the parameters allows for better adversarial attacks and could also disclose sensitive information from the dataset used to construct the network. On the other hand, if the parameters of a network can be recovered, it guarantees the user that the features in the latent spaces can be interpreted. It also provides foundations to obtain formal guarantees on the performances of the network. It is therefore important to characterize the networks whose parameters can be identified and those whose parameters cannot. In this article, we provide a set of conditions on a deep fully-connected feedforward ReLU neural network under which the parameters of the network are uniquely identified-modulo permutation and positive rescaling-from the function it implements on a subset of the input space.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 38 of its 69 references that are in the map. First posted on arXiv 24 Dec 2021.

Builds on

38 of 69 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  8. 2016Deep Learning with Differential Privacycited by 2,728
  9. 2016Adversarial examples in the physical worldcited by 1,769
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  11. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  12. 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
26 more
  1. 2016Universal adversarial perturbationscited by 784
  2. 2017One pixel attack for fooling deep neural networkscited by 582
  3. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  4. 2017Convergence Analysis of Two-layer Neural Networks with ReLU Activationcited by 137
  5. 2017Recovery Guarantees for One-hidden-layer Neural Networkscited by 94
  6. 2019Reverse-Engineering Deep ReLU Networkscited by 83
  7. 2017Learning One-hidden-layer Neural Networks with Landscape Designcited by 81
  8. 2020Cryptanalytic Extraction of Neural Network Modelscited by 68
  9. 2017Adversarial Transformation Networks: Learning to Generate Adversarial Examplescited by 62
  10. 2017Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputscited by 61
  11. 2017Houdini: Fooling Deep Structured Prediction Modelscited by 50
  12. 2018Learning One-hidden-layer ReLU Networks via Gradient Descentcited by 50
  13. 2021A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Networkcited by 47
  14. 2020Lossless Compression of Deep Neural Networkscited by 34
  15. 2018Guaranteed Recovery of One-Hidden-Layer Neural Networks via Cross Entropycited by 22
  16. 2017When is a Convolutional Filter Easy To Learn?cited by 21
  17. 2018Learning One Convolutional Layer with Overlapping Patchescited by 21
  18. 2021An Embedding of ReLU Networks and an Analysis of their Identifiabilitycited by 20
  19. 2017UPSET and ANGRI : Breaking High Performance Image Classifierscited by 19
  20. 2020TRU-NET: A Deep Learning Approach to High Resolution Prediction of Rainfallcited by 19
  21. 2021Scaling Up Exact Neural Network Compression by ReLU Stabilitycited by 19
  22. 2019How degenerate is the parametrization of neural networks with the ReLU activation function?cited by 15
  23. 2013Provable Bounds for Learning Some Deep Representationscited by 14
  24. 2017On the stable recovery of deep structured linear networks under sparsity constraintscited by 5
  25. 2014Provable Methods for Training Neural Networks with Sparse Connectivitycited by 4
  26. 2020A Protection against the Extraction of Neural Network Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 3; 2025: 6; 2026: 9.

24 papers.

Sort:

  1. 2025Symmetry in Neural Network Parameter Spacescited by 36
  2. 2023Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetrycited by 21
  3. 2024Geometry of Lightning Self-Attention: Identifiability and Dimensioncited by 16
  4. 2022Local Identifiability of Deep ReLU Neural Networks: the Theorycited by 12
  5. 2023Copula-Based Deep Survival Models for Dependent Censoringcited by 12
  6. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  7. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  8. 2023Functional Equivalence and Path Connectivity of Reducible Hyperbolic Tangent Networkscited by 6
  9. 2024Dagma-DCE: Interpretable, Non-Parametric Differentiable Causal Discoverycited by 6
  10. 2025The Alexander-Hirschowitz theorem for neurovarietiescited by 6
  11. 2026Most ReLU Networks Admit Identifiable Parameterscited by 6
  12. 2023Exploring the Complexity of Deep Neural Networks through Functional Equivalencecited by 4
12 more
  1. 2025Constraining the outputs of ReLU neural networkscited by 3
  2. 2026On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriorscited by 3
  3. 2026The Symmetries of Three-Layer ReLU Networkscited by 3
  4. 2026On the Epistemic Uncertainty of Overparametrized Neural Networkscited by 2
  5. 2026Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixingcited by 1
  6. 2026Hidden Gauge Controls Feature Specialization in ReLU Networkscited by 1
  7. 2026Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networkscited by 1
  8. 2023SPARLING: Learning Latent Representations with Extremely Sparse Activationscited by 0
  9. 2024Sequencing the Neurome: Towards Scalable Exact Parameter Reconstruction of Black-Box Neural Networkscited by 0
  10. 2025Finite Samples for Shallow Neural Networkscited by 0
  11. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0
  12. 2026Curvature Cryptanalysis of Smooth Transformer Feed-Forward Networkscited by 0