# Modeling Generalization in Machine Learning: A Methodological and Computational Study

arXiv: https://arxiv.org/abs/2006.15680 · HTML: https://mlsanity.com/paper/2006.15680/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theory](https://mlsanity.com/paper/2402.05916/) (2024)
- [Do LLMs Have the Generalization Ability in Conducting Causal Inference?](https://mlsanity.com/paper/2410.11385/) (2024)
- [Model Risk Management for Generative AI In Financial Institutions](https://mlsanity.com/paper/2503.15668/) (2025)
- [Empirical Comparison between Cross-Validation and Mutation-Validation in Model Selection](https://mlsanity.com/paper/2311.14079/) (2023)
- [On Procrustes Contamination in Machine Learning Applications of Geometric Morphometrics](https://mlsanity.com/paper/2601.18448/) (2026)

## Abstract (arXiv)

As machine learning becomes more and more available to the general public, theoretical questions are turning into pressing practical issues. Possibly, one of the most relevant concerns is the assessment of our confidence in trusting machine learning predictions. In many real-world cases, it is of utmost importance to estimate the capabilities of a machine learning algorithm to generalize, i.e., to provide accurate predictions on unseen data, depending on the characteristics of the target problem. In this work, we perform a meta-analysis of 109 publicly-available classification data sets, modeling machine learning generalization as a function of a variety of data set characteristics, ranging from number of samples to intrinsic dimensionality, from class-wise feature skewness to F1 evaluated on test samples falling outside the convex hull of the training set. Experimental results demonstrate the relevance of using the concept of the convex hull of the training data in assessing machine learning generalization, by emphasizing the difference between interpolated and extrapolated predictions. Besides several predictable correlations, we observe unexpectedly weak associations between the generalization ability of machine learning models and all metrics related to dimensionality, thus challenging the common assumption that the textitcurse of dimensionality might impair generalization in machine learning.