# Evolving Scientific Discovery by Unifying Data and Background Knowledge with AI Hilbert

arXiv: https://arxiv.org/abs/2308.09474 · HTML: https://mlsanity.com/paper/2308.09474/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 122 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 30 of 35)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [Discovering physical concepts with neural networks](https://mlsanity.com/paper/1807.10300/) (2018)
- [Hyper-Parameter Optimization: A Review of Algorithms and Applications](https://mlsanity.com/paper/2003.05689/) (2020)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [A Bayesian machine scientist to aid in the solution of challenging scientific problems](https://mlsanity.com/paper/2004.12157/) (2020)
- [Machine Learning in the Search for New Fundamental Physics](https://mlsanity.com/paper/2112.03769/) (2021)
- [JuMP 1.0: Recent improvements to a modeling language for mathematical optimization](https://mlsanity.com/paper/2206.03866/) (2022)
- [Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization](https://mlsanity.com/paper/2206.00176/) (2022)
- [DSOS and SDSOS Optimization: More Tractable Alternatives to Sum of Squares and Semidefinite Optimization](https://mlsanity.com/paper/1706.02586/) (2017)
- [Learning Dynamical Systems with Side Information](https://mlsanity.com/paper/2008.10135/) (2020)
- [A unified approach to mixed-integer optimization problems with logical constraints](https://mlsanity.com/paper/1907.02109/) (2019)
- [Branch-and-Bound Performance Estimation Programming: A Unified Methodology for Constructing Optimal Optimization Methods](https://mlsanity.com/paper/2203.07305/) (2022)
- [Symbolic Regression Driven by Training Data and Prior Knowledge](https://mlsanity.com/paper/2004.11947/) (2020)
- [Mixed-Projection Conic Optimization: A New Paradigm for Modeling Rank Constraints](https://mlsanity.com/paper/2009.10395/) (2020)
- [High-Dimensional Regression with Binary Coefficients. Estimating Squared Error and a Phase Transition](https://mlsanity.com/paper/1701.04455/) (2017)
- [Free Energy Wells and Overlap Gap Property in Sparse PCA](https://mlsanity.com/paper/2006.10689/) (2020)
- [Nonparametric Shape-restricted Regression](https://mlsanity.com/paper/1709.05707/) (2017)
- [A Scalable Algorithm For Sparse Portfolio Selection](https://mlsanity.com/paper/1811.00138/) (2018)
- [Learning dynamic polynomial proofs](https://mlsanity.com/paper/1906.01681/) (2019)
- [Semidefinite approximations of the matrix logarithm](https://mlsanity.com/paper/1705.00812/) (2017)
- [Nonconvex piecewise linear functions: Advanced formulations and simple modeling tools](https://mlsanity.com/paper/1708.00050/) (2017)
- [OKRidge: Scalable Optimal k-Sparse Ridge Regression](https://mlsanity.com/paper/2304.06686/) (2023)
- [Globally Optimal Symbolic Regression](https://mlsanity.com/paper/1710.10720/) (2017)
- [Sum-of-Squares Relaxations for Information Theory and Variational Inference](https://mlsanity.com/paper/2206.13285/) (2022)
- [Shape-Constrained Regression using Sum of Squares Polynomials](https://mlsanity.com/paper/2004.03853/) (2020)
- [Partial facial reduction: simplified, equivalent SDPs via approximations of the PSD cone](https://mlsanity.com/paper/1408.4685/) (2014)

## Built on by (most cited first, 18 of 18)

- [DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively](https://mlsanity.com/paper/2509.26603/) (2025)
- [The Need for Verification in AI-Driven Scientific Discovery](https://mlsanity.com/paper/2509.01398/) (2025)
- [When is a System Discoverable from Data? Discovery Requires Chaos](https://mlsanity.com/paper/2511.08860/) (2025)
- [AI-Newton: A Concept-Driven Physical Law Discovery System without Prior Physical Knowledge](https://mlsanity.com/paper/2504.01538/) (2025)
- [Symplectic Neural Flows for Modeling and Discovery](https://mlsanity.com/paper/2412.16787/) (2024)
- [Verification and Validation for Trustworthy Scientific Machine Learning](https://mlsanity.com/paper/2502.15496/) (2025)
- [Decoding complexity: how machine learning is redefining scientific discovery](https://mlsanity.com/paper/2405.04161/) (2024)
- [Data-driven discovery of governing differential equations across physical systems](https://mlsanity.com/paper/2606.09638/) (2026)
- [Reproducibility: The New Frontier in AI Governance](https://mlsanity.com/paper/2510.11595/) (2025)
- [EGG-SR: Embedding Symbolic Equivalence into Symbolic Regression via Equality Graph](https://mlsanity.com/paper/2511.05849/) (2025)
- [Discover physical concepts and equations with machine learning](https://mlsanity.com/paper/2412.12161/) (2024)
- [Automated Modeling Method for Pathloss Model Discovery](https://mlsanity.com/paper/2505.23383/) (2025)
- [Can AI Follow In Einstein's Footsteps?](https://mlsanity.com/paper/2607.27794/) (2026)
- [The RAIL Principles for Neurosymbolic AI: Reasoning, Assurances, Interfacing and Learning](https://mlsanity.com/paper/2608.04285/) (2026)
- [Bridging the Gap Between Scientific Laws Derived by AI Systems and Canonical Knowledge via Abductive Inference with AI-Noether](https://mlsanity.com/paper/2509.23004/) (2025)
- [ERBench: A Benchmark and Testsuite for Equation Discovery Algorithms](https://mlsanity.com/paper/2606.09276/) (2026)
- [AutoSR: Automatic Symbolic Regression by Searching Research States](https://mlsanity.com/paper/2608.16876/) (2026)
- [Symbolic Neural ODEs: Learning interpretable models from time-series data](https://mlsanity.com/paper/2608.22112/) (2026)

## Abstract (arXiv)

The discovery of scientific formulae that parsimoniously explain natural phenomena and align with existing background theory is a key goal in science. Historically, scientists have derived natural laws by manipulating equations based on existing knowledge, forming new equations, and verifying them experimentally. In recent years, data-driven scientific discovery has emerged as a viable competitor in settings with large amounts of experimental data. Unfortunately, data-driven methods often fail to discover valid laws when data is noisy or scarce. Accordingly, recent works combine regression and reasoning to eliminate formulae inconsistent with background theory. However, the problem of searching over the space of formulae consistent with background theory to find one that best fits the data is not well-solved. We propose a solution to this problem when all axioms and scientific laws are expressible via polynomial equalities and inequalities and argue that our approach is widely applicable. We model notions of minimal complexity using binary variables and logical constraints, solve polynomial optimization problems via mixed-integer linear or semidefinite optimization, and prove the validity of our scientific discoveries in a principled manner using Positivstellensatz certificates. The optimization techniques leveraged in this paper allow our approach to run in polynomial time with fully correct background theory under an assumption that the complexity of our derivation is bounded), or non-deterministic polynomial (NP) time with partially correct background theory. We demonstrate that some famous scientific laws, including Kepler's Third Law of Planetary Motion, the Hagen-Poiseuille Equation, and the Radiated Gravitational Wave Power equation, can be derived in a principled manner from axioms and experimental data.