# Statistical learning theory and Occam's razor: Regularization

arXiv: https://arxiv.org/abs/2608.04049 · HTML: https://mlsanity.com/paper/2608.04049/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 60 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 7 of 7)

- [Deep learning: a statistical viewpoint](https://mlsanity.com/paper/2103.09177/) (2021)
- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation](https://mlsanity.com/paper/2105.14368/) (2021)
- [Minimum Description Length Revisited](https://mlsanity.com/paper/1908.08484/) (2019)
- [The no-free-lunch theorems of supervised learning](https://mlsanity.com/paper/2202.04513/) (2022)
- [Statistical learning theory and Occam's razor: The core argument](https://mlsanity.com/paper/2312.13842/) (2023)
- [Unified Inductive Logic: From Formal Learning to Statistical Inference to Supervised Learning](https://mlsanity.com/paper/2412.02969/) (2024)

## Abstract (arXiv)

The principle of Occam's razor, which instructs us to prefer simplicity in inductive inference, has attracted much scrutiny both in the philosophy of science and in machine learning. In either field, however, a justification for the principle has been elusive. In this paper, building on an earlier "core argument," I spell out a justification from statistical learning theory for the procedure of regularization: for trading off fit for simplicity. The means-ends argument is that in order to profit from theoretical reliability and "what-you-see-is-what-you-get" guarantees, one must implement a certain preference for simplicity over fit. This is a genuine methodological justification, which neither collapses to a purely pragmatic principle that we prefer simplicity for its own sake, nor to an ontological assumption that the truth is simple.