# Information theory and discriminative sampling for model discovery

arXiv: https://arxiv.org/abs/2512.16000 · HTML: https://mlsanity.com/paper/2512.16000/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Builds on (most cited first, 15 of 15)

- [Self-Supervised Exploration via Disagreement](https://mlsanity.com/paper/1906.04161/) (2019)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Weak SINDy For Partial Differential Equations](https://mlsanity.com/paper/2007.02848/) (2020)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control](https://mlsanity.com/paper/2111.10992/) (2021)
- [SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics](https://mlsanity.com/paper/2004.02322/) (2020)
- [Model-Based Active Exploration](https://mlsanity.com/paper/1810.12162/) (2018)
- [PySINDy: A comprehensive Python package for robust sparse system identification](https://mlsanity.com/paper/2111.08481/) (2021)
- [Predictive Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1511.05467/) (2015)
- [Active Learning for Nonlinear System Identification with Guarantees](https://mlsanity.com/paper/2006.10277/) (2020)
- [Active Learning in Robotics: A Review of Control Principles](https://mlsanity.com/paper/2106.13697/) (2021)
- [Noisy-Input Entropy Search for Efficient Robust Bayesian Optimization](https://mlsanity.com/paper/2002.02820/) (2020)
- [A toolkit for data-driven discovery of governing equations in high-noise regimes](https://mlsanity.com/paper/2111.04870/) (2021)
- [Physics-informed active learning with simultaneous weak-form latent space dynamics identification](https://mlsanity.com/paper/2407.00337/) (2024)
- [Data assimilation and discrepancy modeling with shallow recurrent decoders](https://mlsanity.com/paper/2512.01170/) (2025)

## Built on by (most cited first, 3 of 3)

- [SENDAI: A Hierarchical Sparse-measurement, EfficieNt Data AssImilation Framework](https://mlsanity.com/paper/2601.21664/) (2026)
- [Ill-Conditioning in Dictionary-Based Dynamic-Equation Learning: A Systems Biology Case Study](https://mlsanity.com/paper/2603.11330/) (2026)
- [The Dynamics of Discoverability: How Trajectories and Priors Shape Equation Recovery](https://mlsanity.com/paper/2607.18490/) (2026)

## Abstract (arXiv)

Fisher information and Shannon entropy are fundamental tools for understanding and analyzing dynamical systems from complementary perspectives. They can characterize unknown parameters by quantifying the information contained in variables, or measure how different initial trajectories or temporal segments of a trajectory contribute to learning or inferring system dynamics. In this work, we leverage the Fisher Information Matrix (FIM) within the data-driven framework of em sparse identification of nonlinear dynamics (SINDy). We visualize information patterns in chaotic and non-chaotic systems for both single trajectories and multiple initial conditions, demonstrating how information-based analysis can improve sampling efficiency and enhance model performance by prioritizing more informative data. The benefits of statistical bagging are further elucidated through spectral analysis of the FIM. We also illustrate how Fisher information and entropy metrics can promote data efficiency in three scenarios: when only a single trajectory is available, when a tunable control parameter exists, and when multiple trajectories can be freely initialized. As data-driven model discovery continues to gain prominence, principled sampling strategies guided by quantifiable information metrics offer a powerful approach for improving learning efficiency and reducing data requirements.