# Statistical Mechanics of Dynamical System Identification

arXiv: https://arxiv.org/abs/2403.01723 · HTML: https://mlsanity.com/paper/2403.01723/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 100 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control](https://mlsanity.com/paper/2111.10992/) (2021)
- [Statistical physics of inference: Thresholds and algorithms](https://mlsanity.com/paper/1511.02476/) (2015)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoders](https://mlsanity.com/paper/2201.05136/) (2022)
- [Extracting Sparse High-Dimensional Dynamics from Limited Data](https://mlsanity.com/paper/1707.08528/) (2017)
- [On the Convergence of the SINDy Algorithm](https://mlsanity.com/paper/1805.06445/) (2018)
- [Sparse High-Dimensional Regression: Exact Scalable Algorithms and Phase Transitions](https://mlsanity.com/paper/1709.10029/) (2017)
- [Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization](https://mlsanity.com/paper/2206.00176/) (2022)
- [On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression](https://mlsanity.com/paper/2012.01937/) (2020)
- [Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants](https://mlsanity.com/paper/2211.10575/) (2022)
- [Numerical Aspects for Approximating Governing Equations Using Data](https://mlsanity.com/paper/1809.09170/) (2018)
- [Numerical differentiation of noisy data: A unifying multi-objective optimization framework](https://mlsanity.com/paper/2009.01911/) (2020)
- [Direct Estimation of Parameters in ODE Models Using WENDy: Weak-form Estimation of Nonlinear Dynamics](https://mlsanity.com/paper/2302.13271/) (2023)
- [Benchmarking sparse system identification with low-dimensional chaos](https://mlsanity.com/paper/2302.10787/) (2023)
- [A Review of Data-Driven Discovery for Dynamic Systems](https://mlsanity.com/paper/2210.10663/) (2022)
- [A toolkit for data-driven discovery of governing equations in high-noise regimes](https://mlsanity.com/paper/2111.04870/) (2021)
- [Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery](https://mlsanity.com/paper/2301.12649/) (2023)
- [Rapid Bayesian identification of sparse nonlinear dynamics from scarce and noisy data](https://mlsanity.com/paper/2402.15357/) (2024)
- [Data-Driven Discovery of Coarse-Grained Equations](https://mlsanity.com/paper/2002.00790/) (2020)
- [Asymptotic consistency of the WSINDy algorithm in the limit of continuum data](https://mlsanity.com/paper/2211.16000/) (2022)
- [Simultaneous Identification and Denoising of Dynamical Systems](https://mlsanity.com/paper/2203.13837/) (2022)
- [Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inference](https://mlsanity.com/paper/2401.16943/) (2024)
- [A Bayesian Framework for learning governing Partial Differential Equation from Data](https://mlsanity.com/paper/2306.04894/) (2023)
- [Discovering stochastic partial differential equations from limited data using variational Bayes inference](https://mlsanity.com/paper/2306.15873/) (2023)
- [Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels](https://mlsanity.com/paper/2310.05387/) (2023)
- [Statistical mechanical analysis of sparse linear regression as a variable selection problem](https://mlsanity.com/paper/1805.11259/) (2018)

## Built on by (most cited first, 5 of 5)

- [Sparse Identification of Nonlinear Dynamics with Conformal Prediction](https://mlsanity.com/paper/2507.11739/) (2025)
- [Extracting Governing Equations from Latent Dynamics via Multi-View Contrastive Learning](https://mlsanity.com/paper/2606.13260/) (2026)
- [Optimizing Hard Thresholding for Sparse Model Discovery](https://mlsanity.com/paper/2504.20256/) (2025)
- [A joint optimization approach to identifying sparse dynamics using least squares kernel collocation](https://mlsanity.com/paper/2511.18555/) (2025)
- [From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identification](https://mlsanity.com/paper/2512.14404/) (2025)

## Abstract (arXiv)

Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanics approach to analyze sparse equation discovery algorithms, which typically balance data fit and parsimony via hyperparameter tuning. In this framework, statistical mechanics offers tools to analyze the interplay between complexity and fitness similarly to that of entropy and energy in physical systems. To establish this analogy, we define the hyperparameter optimization procedure as a two-level Bayesian inference problem that separates variable selection from coefficient inference and enables the computation of the posterior parameter distribution in closed form. Our approach provides uncertainty quantification, crucial in the low-data limit that is frequently encountered in real-world applications. A key advantage of employing statistical mechanical concepts, such as free energy and the partition function, is to connect the large data limit to thermodynamic limit and characterize the sparsity- and noise-induced phase transitions that delineate correct from incorrect identification. We thus provide a method for closed-loop inference, estimating the noise in a given model and checking if the model is tolerant to that noise amount. This perspective of sparse equation discovery is versatile and can be adapted to various other equation discovery algorithms.