# Critical Sampling for Robust Evolution Operator Learning of Unknown Dynamical Systems

arXiv: https://arxiv.org/abs/2304.07485 · HTML: https://mlsanity.com/paper/2304.07485/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 61 references that are in the map.
- First posted on arXiv 15 Apr 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [Data Driven Governing Equations Approximation Using Deep Neural Networks](https://mlsanity.com/paper/1811.05537/) (2018)
- [Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems](https://mlsanity.com/paper/1912.08177/) (2019)
- [Active Learning for Nonlinear System Identification with Guarantees](https://mlsanity.com/paper/2006.10277/) (2020)
- [Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE](https://mlsanity.com/paper/2006.02493/) (2020)
- [Failure-informed adaptive sampling for PINNs](https://mlsanity.com/paper/2210.00279/) (2022)
- [Extracting Sparse High-Dimensional Dynamics from Limited Data](https://mlsanity.com/paper/1707.08528/) (2017)
- [Data-Driven Deep Learning of Partial Differential Equations in Modal Space](https://mlsanity.com/paper/1910.06948/) (2019)
- [DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm](https://mlsanity.com/paper/2001.07305/) (2020)
- [MALI: A memory efficient and reverse accurate integrator for Neural ODEs](https://mlsanity.com/paper/2102.04668/) (2021)
- [Linear dynamical neural population models through nonlinear embeddings](https://mlsanity.com/paper/1605.08454/) (2016)
- [Thermodynamics-informed graph neural networks](https://mlsanity.com/paper/2203.01874/) (2022)
- [Deep Neural Network Modeling of Unknown Partial Differential Equations in Nodal Space](https://mlsanity.com/paper/2106.03603/) (2021)
- [Numerical Aspects for Approximating Governing Equations Using Data](https://mlsanity.com/paper/1809.09170/) (2018)
- [Robust discovery of partial differential equations in complex situations](https://mlsanity.com/paper/2106.00008/) (2021)
- [Learning reduced systems via deep neural networks with memory](https://mlsanity.com/paper/2003.09451/) (2020)
- [Deep-learning of Parametric Partial Differential Equations from Sparse and Noisy Data](https://mlsanity.com/paper/2005.07916/) (2020)
- [Structure-preserving Method for Reconstructing Unknown Hamiltonian Systems from Trajectory Data](https://mlsanity.com/paper/1905.10396/) (2019)
- [Discovery of Dynamics Using Linear Multistep Methods](https://mlsanity.com/paper/1912.12728/) (2019)
- [On generalized residue network for deep learning of unknown dynamical systems](https://mlsanity.com/paper/2002.02528/) (2020)

## Abstract (arXiv)

Given an unknown dynamical system, what is the minimum number of samples needed for effective learning of its governing laws and accurate prediction of its future evolution behavior, and how to select these critical samples? In this work, we propose to explore this problem based on a design approach. Starting from a small initial set of samples, we adaptively discover critical samples to achieve increasingly accurate learning of the system evolution. One central challenge here is that we do not know the network modeling error since the ground-truth system state is unknown, which is however needed for critical sampling. To address this challenge, we introduce a multi-step reciprocal prediction network where forward and backward evolution networks are designed to learn the temporal evolution behavior in the forward and backward time directions, respectively. Very interestingly, we find that the desired network modeling error is highly correlated with the multi-step reciprocal prediction error, which can be directly computed from the current system state. This allows us to perform a dynamic selection of critical samples from regions with high network modeling errors for dynamical systems. Additionally, a joint spatial-temporal evolution network is introduced which incorporates spatial dynamics modeling into the temporal evolution prediction for robust learning of the system evolution operator with few samples. Our extensive experimental results demonstrate that our proposed method is able to dramatically reduce the number of samples needed for effective learning and accurate prediction of evolution behaviors of unknown dynamical systems by up to hundreds of times.