# Fundamental limits to learning closed-form mathematical models from data

arXiv: https://arxiv.org/abs/2204.02704 · HTML: https://mlsanity.com/paper/2204.02704/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 43 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 8 of 8)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Statistical physics of inference: Thresholds and algorithms](https://mlsanity.com/paper/1511.02476/) (2015)
- [Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models](https://mlsanity.com/paper/1708.03395/) (2017)
- [The role of regularization in classification of high-dimensional noisy Gaussian mixture](https://mlsanity.com/paper/2002.11544/) (2020)
- [A Bayesian machine scientist to aid in the solution of challenging scientific problems](https://mlsanity.com/paper/2004.12157/) (2020)
- [Fundamental Limits of Weak Recovery with Applications to Phase Retrieval](https://mlsanity.com/paper/1708.05932/) (2017)
- [Learning models of quantum systems from experiments](https://mlsanity.com/paper/2002.06169/) (2020)
- [Consistencies and inconsistencies between model selection and link prediction in networks](https://mlsanity.com/paper/1705.07967/) (2017)

## Built on by (most cited first, 12 of 12)

- [Discovering Causal Relations and Equations from Data](https://mlsanity.com/paper/2305.13341/) (2023)
- [Generative learning for nonlinear dynamics](https://mlsanity.com/paper/2311.04128/) (2023)
- [Human mobility is well described by closed-form gravity-like models learned automatically from data](https://mlsanity.com/paper/2312.11281/) (2023)
- [Is the Machine Smarter than the Theorist: Deriving Formulas for Particle Kinematics with Symbolic Regression](https://mlsanity.com/paper/2211.08420/) (2022)
- [A Triumvirate of AI Driven Theoretical Discovery](https://mlsanity.com/paper/2405.19973/) (2024)
- [Active Learning in Symbolic Regression with Physical Constraints](https://mlsanity.com/paper/2305.10379/) (2023)
- [Decoding complexity: how machine learning is redefining scientific discovery](https://mlsanity.com/paper/2405.04161/) (2024)
- [Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores](https://mlsanity.com/paper/2512.01080/) (2025)
- [Discovering stochastic dynamical equations from biological time series data](https://mlsanity.com/paper/2205.02645/) (2022)
- [Bayesian symbolic regression: Automated equation discovery from a physicists' perspective](https://mlsanity.com/paper/2507.19540/) (2025)
- [Input-schema identifiability limits in physics-informed surrogates for mechanics-governed flow](https://mlsanity.com/paper/2606.20655/) (2026)
- [Learning the Geometry of Admissible Hypotheses through Inductive Bias in Training Distributions](https://mlsanity.com/paper/2608.31028/) (2026)

## Abstract (arXiv)

Given a finite and noisy dataset generated with a closed-form mathematical model, when is it possible to learn the true generating model from the data alone? This is the question we investigate here. We show that this model-learning problem displays a transition from a low-noise phase in which the true model can be learned, to a phase in which the observation noise is too high for the true model to be learned by any method. Both in the low-noise phase and in the high-noise phase, probabilistic model selection leads to optimal generalization to unseen data. This is in contrast to standard machine learning approaches, including artificial neural networks, which in this particular problem are limited, in the low-noise phase, by their ability to interpolate. In the transition region between the learnable and unlearnable phases, generalization is hard for all approaches including probabilistic model selection.