# Learning the hypotheses space from data through a U-curve algorithm

arXiv: https://arxiv.org/abs/2109.03866 · HTML: https://mlsanity.com/paper/2109.03866/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 64 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 6 of 6)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks](https://mlsanity.com/paper/1805.12076/) (2018)
- [Model Selection Techniques -- An Overview](https://mlsanity.com/paper/1810.09583/) (2018)

## Abstract (arXiv)

This paper proposes a data-driven systematic, consistent and non-exhaustive approach to Model Selection, that is an extension of the classical agnostic PAC learning model. In this approach, learning problems are modeled not only by a hypothesis space H, but also by a Learning Space mathbbL(H), a poset of subspaces of H, which covers H and satisfies a property regarding the VC dimension of related subspaces, that is a suitable algebraic search space for Model Selection algorithms. Our main contributions are a data-driven general learning algorithm to perform implicitly regularized Model Selection on mathbbL(H) and a framework under which one can, theoretically, better estimate a target hypothesis with a given sample size by properly modeling mathbbL(H) and employing high computational power. A remarkable consequence of this approach are conditions under which a non-exhaustive search of mathbbL(H) can return an optimal solution. The results of this paper lead to a practical property of Machine Learning, that the lack of experimental data may be mitigated by a high computational capacity. In a context of continuous popularization of computational power, this property may help understand why Machine Learning has become so important, even where data is expensive and hard to get.