# Towards a mathematical framework to inform Neural Network modelling via Polynomial Regression

arXiv: https://arxiv.org/abs/2102.03865 · HTML: https://mlsanity.com/paper/2102.03865/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 7 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [Towards Explainable Deep Neural Networks (xDNN)](https://mlsanity.com/paper/1912.02523/) (2019)
- [Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications](https://mlsanity.com/paper/2003.07631/) (2020)
- [Polynomial Regression As an Alternative to Neural Nets](https://mlsanity.com/paper/1806.06850/) (2018)
- [Optimal Uncertainty-guided Neural Network Training](https://mlsanity.com/paper/1912.12761/) (2019)

## Built on by (most cited first, 6 of 6)

- [NN2Poly: A polynomial representation for deep feed-forward artificial neural networks](https://mlsanity.com/paper/2112.11397/) (2021)
- [Activation Functions Not To Active: A Plausible Theory on Interpreting Neural Networks](https://mlsanity.com/paper/2305.00663/) (2023)
- [HOPE: High-order Polynomial Expansion of Black-box Neural Networks](https://mlsanity.com/paper/2307.08192/) (2023)
- [Uncertainty Aware Neural Network from Similarity and Sensitivity](https://mlsanity.com/paper/2304.14925/) (2023)
- [Neural lasso: a unifying approach of lasso and neural networks](https://mlsanity.com/paper/2309.03770/) (2023)
- [nn2poly: An R Package for Converting Neural Networks into Interpretable Polynomials](https://mlsanity.com/paper/2406.01588/) (2024)

## Abstract (arXiv)

Even when neural networks are widely used in a large number of applications, they are still considered as black boxes and present some difficulties for dimensioning or evaluating their prediction error. This has led to an increasing interest in the overlapping area between neural networks and more traditional statistical methods, which can help overcome those problems. In this article, a mathematical framework relating neural networks and polynomial regression is explored by building an explicit expression for the coefficients of a polynomial regression from the weights of a given neural network, using a Taylor expansion approach. This is achieved for single hidden layer neural networks in regression problems. The validity of the proposed method depends on different factors like the distribution of the synaptic potentials or the chosen activation function. The performance of this method is empirically tested via simulation of synthetic data generated from polynomials to train neural networks with different structures and hyperparameters, showing that almost identical predictions can be obtained when certain conditions are met. Lastly, when learning from polynomial generated data, the proposed method produces polynomials that approximate correctly the data locally.