# Investigating Statistical Inference and Covariate Effects in Shallow Neural Networks

arXiv: https://arxiv.org/abs/2311.08139 · HTML: https://mlsanity.com/paper/2311.08139/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 6 of 6)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay](https://mlsanity.com/paper/1803.09820/) (2018)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)
- [A Statistical-Modelling Approach to Feedforward Neural Network Model Selection](https://mlsanity.com/paper/2207.04248/) (2022)

## Built on by (most cited first, 1 of 1)

- [Measuring Neural Network Complexity via Effective Degrees of Freedom](https://mlsanity.com/paper/2602.13442/) (2026)

## Abstract (arXiv)

Feedforward neural networks (FNNs) are typically viewed as pure prediction algorithms, and their strong predictive performance has led to their use in many machine-learning applications. However, their flexibility comes with an interpretability trade-off; thus, FNNs have been historically less popular among statisticians. Nevertheless, for suitably parsimonious shallow FNNs, classical statistical theory, such as significance testing and uncertainty quantification, may still provide useful regression-style summaries. Supplementing FNNs with methods of statistical inference, and covariate-effect visualisations, can shift the focus away from black-box prediction and move FNNs towards traditional statistical models. This can allow for more inferential analysis, and, hence, make FNNs more accessible within the statistical-modelling context. We investigate covariate-level Wald testing in the context of penalised FNNs, and also propose covariate-effect plots that emulate regression coefficients. Simulation studies are used to extensively investigate the performance of Wald-based inference, with particular emphasis on when this approach performs well. This statistical-based approach to neural networks is demonstrated through an application to insurance data.