# The Morgan-Pitman Test of Equality of Variances and its Application to Machine Learning Model Evaluation and Selection

arXiv: https://arxiv.org/abs/2509.12185 · HTML: https://mlsanity.com/paper/2509.12185/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 25 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 1 of 1)

- [Significance Tests for Neural Networks](https://mlsanity.com/paper/1902.06021/) (2019)

## Abstract (arXiv)

Model selection in non-linear models often prioritizes performance metrics over statistical tests, limiting the ability to account for sampling variability. We propose the use of a statistical test to assess the equality of variances in forecasting errors. The test builds upon the classic Morgan-Pitman approach, incorporating enhancements to ensure robustness against data with heavy-tailed distributions or outliers with high variance, plus a strategy to make residuals from machine learning models statistically independent. Through a series of simulations and real-world data applications, we demonstrate the test's effectiveness and practical utility, offering a reliable tool for model evaluation and selection in diverse contexts.