# Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery

arXiv: https://arxiv.org/abs/2301.12649 · HTML: https://mlsanity.com/paper/2301.12649/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 95 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 17 of 17)

- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control](https://mlsanity.com/paper/2111.10992/) (2021)
- [SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics](https://mlsanity.com/paper/2004.02322/) (2020)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [PySINDy: A comprehensive Python package for robust sparse system identification](https://mlsanity.com/paper/2111.08481/) (2021)
- [Sparse learning of stochastic dynamic equations](https://mlsanity.com/paper/1712.02432/) (2017)
- [Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoders](https://mlsanity.com/paper/2201.05136/) (2022)
- [On the Convergence of the SINDy Algorithm](https://mlsanity.com/paper/1805.06445/) (2018)
- [Robust Regression via Hard Thresholding](https://mlsanity.com/paper/1506.02428/) (2015)
- [A Unified Framework for Sparse Relaxed Regularized Regression: SR3](https://mlsanity.com/paper/1807.05411/) (2018)
- [False Discoveries Occur Early on the Lasso Path](https://mlsanity.com/paper/1511.01957/) (2015)
- [Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants](https://mlsanity.com/paper/2211.10575/) (2022)
- [On the Non-asymptotic and Sharp Lower Tail Bounds of Random Variables](https://mlsanity.com/paper/1810.09006/) (2018)
- [Bayesian Bootstrap Spike-and-Slab LASSO](https://mlsanity.com/paper/2011.14279/) (2020)
- [Can we trust the bootstrap in high-dimension?](https://mlsanity.com/paper/1608.00696/) (2016)
- [Bayesian data-driven discovery of partial differential equations with variable coefficients](https://mlsanity.com/paper/2102.01432/) (2021)
- [Loss-guided Stability Selection](https://mlsanity.com/paper/2202.04956/) (2022)

## Built on by (most cited first, 15 of 15)

- [Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networks](https://mlsanity.com/paper/2501.13329/) (2025)
- [Benchmarking sparse system identification with low-dimensional chaos](https://mlsanity.com/paper/2302.10787/) (2023)
- [Rapid Bayesian identification of sparse nonlinear dynamics from scarce and noisy data](https://mlsanity.com/paper/2402.15357/) (2024)
- [HyperSINDy: Deep Generative Modeling of Nonlinear Stochastic Governing Equations](https://mlsanity.com/paper/2310.04832/) (2023)
- [Statistical Mechanics of Dynamical System Identification](https://mlsanity.com/paper/2403.01723/) (2024)
- [Mesh-free sparse identification of nonlinear dynamics](https://mlsanity.com/paper/2505.16058/) (2025)
- [T-SHRED: Symbolic Regression for Regularization and Model Discovery with Transformer Shallow Recurrent Decoders](https://mlsanity.com/paper/2506.15881/) (2025)
- [Sparse Identification of Nonlinear Dynamics with Conformal Prediction](https://mlsanity.com/paper/2507.11739/) (2025)
- [PySHRED: A Python package for SHallow REcurrent Decoding for sparse sensing, model reduction and scientific discovery](https://mlsanity.com/paper/2507.20954/) (2025)
- [Overcoming error-in-variable problem in data-driven model discovery by orthogonal distance regression](https://mlsanity.com/paper/2507.23426/) (2025)
- [Comparison of Single- and Multi- Objective Optimization Quality for Evolutionary Equation Discovery](https://mlsanity.com/paper/2306.17038/) (2023)
- [Exact identification of nonlinear dynamical systems by Trimmed Lasso](https://mlsanity.com/paper/2308.01891/) (2023)
- [A joint optimization approach to identifying sparse dynamics using least squares kernel collocation](https://mlsanity.com/paper/2511.18555/) (2025)
- [From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identification](https://mlsanity.com/paper/2512.14404/) (2025)
- [PSI-SINDy: Post-Selection Inference for Sparse Identification of Nonlinear Dynamics](https://mlsanity.com/paper/2610.11486/) (2026)

## Abstract (arXiv)

Sparse model identification enables nonlinear dynamical system discovery from data. However, the control of false discoveries for sparse model identification is challenging, especially in the low-data and high-noise limit. In this paper, we perform a theoretical study on ensemble sparse model discovery, which shows empirical success in terms of accuracy and robustness to noise. In particular, we analyse the bootstrapping-based sequential thresholding least-squares estimator. We show that this bootstrapping-based ensembling technique can perform a provably correct variable selection procedure with an exponential convergence rate of the error rate. In addition, we show that the ensemble sparse model discovery method can perform computationally efficient uncertainty estimation, compared to expensive Bayesian uncertainty quantification methods via MCMC. We demonstrate the convergence properties and connection to uncertainty quantification in various numerical studies on synthetic sparse linear regression and sparse model discovery. The experiments on sparse linear regression support that the bootstrapping-based sequential thresholding least-squares method has better performance for sparse variable selection compared to LASSO, thresholding least-squares, and bootstrapping-based LASSO. In the sparse model discovery experiment, we show that the bootstrapping-based sequential thresholding least-squares method can provide valid uncertainty quantification, converging to a delta measure centered around the true value with increased sample sizes. Finally, we highlight the improved robustness to hyperparameter selection under shifting noise and sparsity levels of the bootstrapping-based sequential thresholding least-squares method compared to other sparse regression methods.