# Improved identification accuracy in equation learning via comprehensive boldsymbolR2-elimination and Bayesian model selection

arXiv: https://arxiv.org/abs/2311.13265 · HTML: https://mlsanity.com/paper/2311.13265/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 48 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 20 of 20)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Weak SINDy For Partial Differential Equations](https://mlsanity.com/paper/2007.02848/) (2020)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [PySINDy: A comprehensive Python package for robust sparse system identification](https://mlsanity.com/paper/2111.08481/) (2021)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [Extracting Sparse High-Dimensional Dynamics from Limited Data](https://mlsanity.com/paper/1707.08528/) (2017)
- [A unified sparse optimization framework to learn parsimonious physics-informed models from data](https://mlsanity.com/paper/1906.10612/) (2019)
- [Automatic Differentiation to Simultaneously Identify Nonlinear Dynamics and Extract Noise Probability Distributions from Data](https://mlsanity.com/paper/2009.08810/) (2020)
- [A Unified Framework for Sparse Relaxed Regularized Regression: SR3](https://mlsanity.com/paper/1807.05411/) (2018)
- [False Discoveries Occur Early on the Lasso Path](https://mlsanity.com/paper/1511.01957/) (2015)
- [Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization](https://mlsanity.com/paper/2206.00176/) (2022)
- [On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression](https://mlsanity.com/paper/2012.01937/) (2020)
- [Sparse Identification of Nonlinear Dynamical Systems via Reweighted ℓ1-regularized Least Squares](https://mlsanity.com/paper/2005.13232/) (2020)
- [Bayesian Evidence and Model Selection](https://mlsanity.com/paper/1411.3013/) (2014)
- [A Pliable Lasso](https://mlsanity.com/paper/1712.00484/) (2017)

## Abstract (arXiv)

In the field of equation learning, exhaustively considering all possible equations derived from a basis function dictionary is infeasible. Sparse regression and greedy algorithms have emerged as popular approaches to tackle this challenge. However, the presence of multicollinearity poses difficulties for sparse regression techniques, and greedy steps may inadvertently exclude terms of the true equation, leading to reduced identification accuracy. In this article, we present an approach that strikes a balance between comprehensiveness and efficiency in equation learning. Inspired by stepwise regression, our approach combines the coefficient of determination, R2, and the Bayesian model evidence, p(boldsymbol y|mathcal M), in a novel way. Our procedure is characterized by a comprehensive search with just a minor reduction of the model space at each iteration step. With two flavors of our approach and the adoption of p(boldsymbol y|mathcal M) for bi-directional stepwise regression, we present a total of three new avenues for equation learning. Through three extensive numerical experiments involving random polynomials and dynamical systems, we compare our approach against four state-of-the-art methods and two standard approaches. The results demonstrate that our comprehensive search approach surpasses all other methods in terms of identification accuracy. In particular, the second flavor of our approach establishes an efficient overfitting penalty solely based on R2, which achieves highest rates of exact equation recovery.