mlsanityData to 7 Oct 2026

Statistical Mechanics of Dynamical System Identification

arXiv 2403.01723 ↗First posted 4 Mar 2024cond-mat.stat-mech cs.LG math.OCcited by 5

Not on the map. Why?

Abstract (arXiv)

Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanics approach to analyze sparse equation discovery algorithms, which typically balance data fit and parsimony via hyperparameter tuning. In this framework, statistical mechanics offers tools to analyze the interplay between complexity and fitness similarly to that of entropy and energy in physical systems. To establish this analogy, we define the hyperparameter optimization procedure as a two-level Bayesian inference problem that separates variable selection from coefficient inference and enables the computation of the posterior parameter distribution in closed form. Our approach provides uncertainty quantification, crucial in the low-data limit that is frequently encountered in real-world applications. A key advantage of employing statistical mechanical concepts, such as free energy and the partition function, is to connect the large data limit to thermodynamic limit and characterize the sparsity- and noise-induced phase transitions that delineate correct from incorrect identification. We thus provide a method for closed-loop inference, estimating the noise in a given model and checking if the model is tolerant to that noise amount. This perspective of sparse equation discovery is versatile and can be adapted to various other equation discovery algorithms.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 30 of its 100 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

30 of 100 references.

References · 30

  1. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  2. 2018Learning Equations for Extrapolation and Controlcited by 151
  3. 2020AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularitycited by 140
  4. 2017Sparse identification of nonlinear dynamics for model predictive control in the low-data limitcited by 139
  5. 2021Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and controlcited by 134
  6. 2015Statistical physics of inference: Thresholds and algorithmscited by 128
  7. 2021Chaos as an interpretable benchmark for forecasting and data-driven modellingcited by 87
  8. 2022Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoderscited by 58
  9. 2017Extracting Sparse High-Dimensional Dynamics from Limited Datacited by 50
  10. 2018On the Convergence of the SINDy Algorithmcited by 49
  11. 2017Sparse High-Dimensional Regression: Exact Scalable Algorithms and Phase Transitionscited by 43
  12. 2022Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimizationcited by 29
18 more
  1. 2020On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regressioncited by 24
  2. 2022Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constantscited by 24
  3. 2018Numerical Aspects for Approximating Governing Equations Using Datacited by 21
  4. 2023Direct Estimation of Parameters in ODE Models Using WENDy: Weak-form Estimation of Nonlinear Dynamicscited by 20
  5. 2020Numerical differentiation of noisy data: A unifying multi-objective optimization frameworkcited by 19
  6. 2023Benchmarking sparse system identification with low-dimensional chaoscited by 19
  7. 2022A Review of Data-Driven Discovery for Dynamic Systemscited by 18
  8. 2021A toolkit for data-driven discovery of governing equations in high-noise regimescited by 15
  9. 2023Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discoverycited by 15
  10. 2024Rapid Bayesian identification of sparse nonlinear dynamics from scarce and noisy datacited by 15
  11. 2020Data-Driven Discovery of Coarse-Grained Equationscited by 14
  12. 2022Asymptotic consistency of the WSINDy algorithm in the limit of continuum datacited by 14
  13. 2022Simultaneous Identification and Denoising of Dynamical Systemscited by 10
  14. 2024Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inferencecited by 9
  15. 2023A Bayesian Framework for learning governing Partial Differential Equation from Datacited by 8
  16. 2023Discovering stochastic partial differential equations from limited data using variational Bayes inferencecited by 6
  17. 2023Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernelscited by 5
  18. 2018Statistical mechanical analysis of sparse linear regression as a variable selection problemcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Sparse Identification of Nonlinear Dynamics with Conformal Predictioncited by 3
  2. 2026Extracting Governing Equations from Latent Dynamics via Multi-View Contrastive Learningcited by 1
  3. 2025Optimizing Hard Thresholding for Sparse Model Discoverycited by 0
  4. 2025A joint optimization approach to identifying sparse dynamics using least squares kernel collocationcited by 0
  5. 2025From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identificationcited by 0