# A Statistical-Modelling Approach to Feedforward Neural Network Model Selection

arXiv: https://arxiv.org/abs/2207.04248 · HTML: https://mlsanity.com/paper/2207.04248/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 35 references that are in the map.
- First posted on arXiv 9 Jul 2022.

## Builds on (most cited first, 4 of 4)

- [Deep Learning for Anomaly Detection: A Review](https://mlsanity.com/paper/2007.02500/) (2020)
- [A Primer on Neural Network Models for Natural Language Processing](https://mlsanity.com/paper/1510.00726/) (2015)
- [Semi-Structured Distributional Regression -- Extending Structured Additive Models by Arbitrary Deep Neural Networks and Data Modalities](https://mlsanity.com/paper/2002.05777/) (2020)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)

## Built on by (most cited first, 5 of 5)

- [Comparison of generalised additive models and neural networks in applications: A systematic review](https://mlsanity.com/paper/2510.24601/) (2025)
- [Investigating Statistical Inference and Covariate Effects in Shallow Neural Networks](https://mlsanity.com/paper/2311.08139/) (2023)
- [A data augmentation strategy for deep neural networks with application to epidemic modelling](https://mlsanity.com/paper/2502.21033/) (2025)
- [Augmented data and neural networks for robust epidemic forecasting: application to COVID-19 in Italy](https://mlsanity.com/paper/2510.09192/) (2025)
- [A Statistical Framework for Model Selection in LSTM Networks](https://mlsanity.com/paper/2506.06840/) (2025)

## Abstract (arXiv)

Feedforward neural networks (FNNs) can be viewed as non-linear regression models, where covariates enter the model through a combination of weighted summations and non-linear functions. Although these models have some similarities to the approaches used within statistical modelling, the majority of neural network research has been conducted outside of the field of statistics. This has resulted in a lack of statistically-based methodology, and, in particular, there has been little emphasis on model parsimony. Determining the input layer structure is analogous to variable selection, while the structure for the hidden layer relates to model complexity. In practice, neural network model selection is often carried out by comparing models using out-of-sample performance. However, in contrast, the construction of an associated likelihood function opens the door to information-criteria-based variable and architecture selection. A novel model selection method, which performs both input- and hidden-node selection, is proposed using the Bayesian information criterion (BIC) for FNNs. The choice of BIC over out-of-sample performance as the model selection objective function leads to an increased probability of recovering the true model, while parsimoniously achieving favourable out-of-sample performance. Simulation studies are used to evaluate and justify the proposed method, and applications on real data are investigated.