# Optimizing Hard Thresholding for Sparse Model Discovery

arXiv: https://arxiv.org/abs/2504.20256 · HTML: https://mlsanity.com/paper/2504.20256/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Builds on (most cited first, 24 of 24)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Weak SINDy For Partial Differential Equations](https://mlsanity.com/paper/2007.02848/) (2020)
- [Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control](https://mlsanity.com/paper/2111.10992/) (2021)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Sparse learning of stochastic dynamic equations](https://mlsanity.com/paper/1712.02432/) (2017)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Extracting Sparse High-Dimensional Dynamics from Limited Data](https://mlsanity.com/paper/1707.08528/) (2017)
- [On the Convergence of the SINDy Algorithm](https://mlsanity.com/paper/1805.06445/) (2018)
- [Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems](https://mlsanity.com/paper/2011.10616/) (2020)
- [Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants](https://mlsanity.com/paper/2211.10575/) (2022)
- [Numerical differentiation of noisy data: A unifying multi-objective optimization framework](https://mlsanity.com/paper/2009.01911/) (2020)
- [A Review of Data-Driven Discovery for Dynamic Systems](https://mlsanity.com/paper/2210.10663/) (2022)
- [SINDy with Control: A Tutorial](https://mlsanity.com/paper/2108.13404/) (2021)
- [ADAM-SINDy: An Efficient Optimization Framework for Parameterized Nonlinear Dynamical System Identification](https://mlsanity.com/paper/2410.16528/) (2024)
- [Random Feature Models for Learning Interacting Dynamical Systems](https://mlsanity.com/paper/2212.05591/) (2022)
- [Time-Series Forecasting, Knowledge Distillation, and Refinement within a Multimodal PDE Foundation Model](https://mlsanity.com/paper/2409.11609/) (2024)
- [BINDy -- Bayesian identification of nonlinear dynamics with reversible-jump Markov-chain Monte-Carlo](https://mlsanity.com/paper/2408.08062/) (2024)
- [Statistical Mechanics of Dynamical System Identification](https://mlsanity.com/paper/2403.01723/) (2024)
- [Multiphysics discovery with moving boundaries using Ensemble SINDy and Peridynamic Differential Operator](https://mlsanity.com/paper/2303.15631/) (2023)
- [The risk of bias in denoising methods](https://mlsanity.com/paper/2201.09351/) (2022)

## Abstract (arXiv)

Many model selection algorithms rely on sparse dictionary learning to provide interpretable and physics-based governing equations. The optimization algorithms typically use a hard thresholding process to enforce sparse activations in the model coefficients by removing library elements from consideration. By introducing an annealing scheme that reactivates a fraction of the removed terms with a cooling schedule, we are able to improve the performance of these sparse learning algorithms. We concentrate on two approaches to the optimization, SINDy, and an alternative using hard thresholding pursuit. We see in both cases that annealing can improve model accuracy. The effectiveness of annealing is demonstrated through comparisons on several nonlinear systems pulled from convective flows, excitable systems, and population dynamics. Finally we apply these algorithms to experimental data for projectile motion.