# A Bayesian machine scientist to aid in the solution of challenging scientific problems

arXiv: https://arxiv.org/abs/2004.12157 · HTML: https://mlsanity.com/paper/2004.12157/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2020.

## Built on by (most cited first, 30 of 61)

- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [ODEFormer: Symbolic Regression of Dynamical Systems with Transformers](https://mlsanity.com/paper/2310.05573/) (2023)
- [Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws](https://mlsanity.com/paper/2303.03192/) (2023)
- [Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl](https://mlsanity.com/paper/2305.01582/) (2023)
- [Discovering Causal Relations and Equations from Data](https://mlsanity.com/paper/2305.13341/) (2023)
- [Automated Scientific Discovery: From Equation Discovery to Autonomous Discovery Systems](https://mlsanity.com/paper/2305.02251/) (2023)
- [Evolving Scientific Discovery by Unifying Data and Background Knowledge with AI Hilbert](https://mlsanity.com/paper/2308.09474/) (2023)
- [Simplifying Polylogarithms with Machine Learning](https://mlsanity.com/paper/2206.04115/) (2022)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Efficient Generator of Mathematical Expressions for Symbolic Regression](https://mlsanity.com/paper/2302.09893/) (2023)
- [Gravity-Bench-v1: A Benchmark on Gravitational Physics Discovery for Agents](https://mlsanity.com/paper/2501.18411/) (2025)
- [Fundamental limits to learning closed-form mathematical models from data](https://mlsanity.com/paper/2204.02704/) (2022)
- [Catch-22s of reservoir computing](https://mlsanity.com/paper/2210.10211/) (2022)
- [Human mobility is well described by closed-form gravity-like models learned automatically from data](https://mlsanity.com/paper/2312.11281/) (2023)
- [Priors for symbolic regression](https://mlsanity.com/paper/2304.06333/) (2023)
- [Symbolic Regression via Control Variable Genetic Programming](https://mlsanity.com/paper/2306.08057/) (2023)
- [AI Descartes: Combining Data and Theory for Derivable Scientific Discovery](https://mlsanity.com/paper/2109.01634/) (2021)
- [Is the Machine Smarter than the Theorist: Deriving Formulas for Particle Kinematics with Symbolic Regression](https://mlsanity.com/paper/2211.08420/) (2022)
- [The Need for Verification in AI-Driven Scientific Discovery](https://mlsanity.com/paper/2509.01398/) (2025)
- [Augmenting astrophysical scaling relations with machine learning: application to reducing the Sunyaev-Zeldovich flux-mass scatter](https://mlsanity.com/paper/2201.01305/) (2022)
- [AI-Assisted Discovery of Quantitative and Formal Models in Social Science](https://mlsanity.com/paper/2210.00563/) (2022)
- [Optimal Inflationary Potentials](https://mlsanity.com/paper/2310.16786/) (2023)
- [Using generative AI to investigate medical imagery models and datasets](https://mlsanity.com/paper/2306.00985/) (2023)
- [On the definition and importance of interpretability in scientific machine learning](https://mlsanity.com/paper/2505.13510/) (2025)
- [Scaling Up Unbiased Search-based Symbolic Regression](https://mlsanity.com/paper/2506.19626/) (2025)
- [Racing Control Variable Genetic Programming for Symbolic Regression](https://mlsanity.com/paper/2309.07934/) (2023)
- [Exploring the Truth and Beauty of Theory Landscapes with Machine Learning](https://mlsanity.com/paper/2401.11513/) (2024)
- [Active Learning in Symbolic Regression with Physical Constraints](https://mlsanity.com/paper/2305.10379/) (2023)
- [Bayesian Symbolic Regression via Posterior Sampling](https://mlsanity.com/paper/2512.10849/) (2025)

## Abstract (arXiv)

Closed-form, interpretable mathematical models have been instrumental for advancing our understanding of the world; with the data revolution, we may now be in a position to uncover new such models for many systems from physics to the social sciences. However, to deal with increasing amounts of data, we need "machine scientists" that are able to extract these models automatically from data. Here, we introduce a Bayesian machine scientist, which establishes the plausibility of models using explicit approximations to the exact marginal posterior over models and establishes its prior expectations about models by learning from a large empirical corpus of mathematical expressions. It explores the space of models using Markov chain Monte Carlo. We show that this approach uncovers accurate models for synthetic and real data and provides out-of-sample predictions that are more accurate than those of existing approaches and of other nonparametric methods.