# Statistical learning theory and Occam's razor: The core argument

arXiv: https://arxiv.org/abs/2312.13842 · HTML: https://mlsanity.com/paper/2312.13842/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 81 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [Deep learning: a statistical viewpoint](https://mlsanity.com/paper/2103.09177/) (2021)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation](https://mlsanity.com/paper/2105.14368/) (2021)
- [The Modern Mathematics of Deep Learning](https://mlsanity.com/paper/2105.04026/) (2021)
- [A Theory of Universal Learning](https://mlsanity.com/paper/2011.04483/) (2020)
- [Fast rates in statistical and online learning](https://mlsanity.com/paper/1507.02592/) (2015)
- [The no-free-lunch theorems of supervised learning](https://mlsanity.com/paper/2202.04513/) (2022)
- [On characterizations of learnability with computable learners](https://mlsanity.com/paper/2202.05041/) (2022)

## Built on by (most cited first, 5 of 5)

- [The challenge of uncertainty quantification of large language models in medicine](https://mlsanity.com/paper/2504.05278/) (2025)
- [The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models](https://mlsanity.com/paper/2510.23191/) (2025)
- [A Plea for History and Philosophy of Statistics and Machine Learning](https://mlsanity.com/paper/2506.22236/) (2025)
- [Benign interpolation and Occam's razor](https://mlsanity.com/paper/2608.03386/) (2026)
- [Statistical learning theory and Occam's razor: Regularization](https://mlsanity.com/paper/2608.04049/) (2026)

## Abstract (arXiv)

Statistical learning theory is often associated with the principle of Occam's razor, which recommends a simplicity preference in inductive inference. This paper distills the core argument for simplicity obtainable from statistical learning theory, built on the theory's central learning guarantee for the method of empirical risk minimization. This core "means-ends" argument is that a simpler hypothesis class or inductive model is better because it has better learning guarantees; however, these guarantees are model-relative and so the theoretical push towards simplicity is checked by our prior knowledge.