# LEADS: Learning Dynamical Systems that Generalize Across Environments

arXiv: https://arxiv.org/abs/2106.04546 · HTML: https://mlsanity.com/paper/2106.04546/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport](https://mlsanity.com/paper/1805.09545/) (2018)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Uniform convergence may be unable to explain generalization in deep learning](https://mlsanity.com/paper/1902.04742/) (2019)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting](https://mlsanity.com/paper/2010.04456/) (2020)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [Learning Dynamical Systems from Partial Observations](https://mlsanity.com/paper/1902.11136/) (2019)
- [ODE2VAE: Deep generative second order ODEs with Bayesian neural networks](https://mlsanity.com/paper/1905.10994/) (2019)
- [A Simple Neural Attentive Meta-Learner](https://mlsanity.com/paper/1707.03141/) (2017)
- [Deep learning-based reduced order models in cardiac electrophysiology](https://mlsanity.com/paper/2006.03040/) (2020)
- [Identifying Physical Law of Hamiltonian Systems via Meta-Learning](https://mlsanity.com/paper/2102.11544/) (2021)

## Built on by (most cited first, 30 of 40)

- [Generalizing to New Physical Systems via Context-Informed Dynamics Model](https://mlsanity.com/paper/2202.01889/) (2022)
- [Out-of-Domain Generalization in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2402.18377/) (2024)
- [Partial Differential Equations Meet Deep Neural Networks: A Survey](https://mlsanity.com/paper/2211.05567/) (2022)
- [Generalization bounds for neural ordinary differential equations and deep residual networks](https://mlsanity.com/paper/2305.06648/) (2023)
- [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025)
- [Zebra: In-Context Generative Pretraining for Solving Parametric PDEs](https://mlsanity.com/paper/2410.03437/) (2024)
- [Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data](https://mlsanity.com/paper/2410.04814/) (2024)
- [Masked Autoencoders are PDE Learners](https://mlsanity.com/paper/2403.17728/) (2024)
- [Strategies for Pretraining Neural Operators](https://mlsanity.com/paper/2406.08473/) (2024)
- [Interpretable Meta-Learning of Physical Systems](https://mlsanity.com/paper/2312.00477/) (2023)
- [MetaNO: How to Transfer Your Knowledge on Learning Hidden Physics](https://mlsanity.com/paper/2301.12095/) (2023)
- [Test-time Generalization for Physics through Neural Operator Splitting](https://mlsanity.com/paper/2602.00884/) (2026)
- [Neural Context Flows for Meta-Learning of Dynamical Systems](https://mlsanity.com/paper/2405.02154/) (2024)
- [Meta-Dynamical State Space Models for Integrative Neural Data Analysis](https://mlsanity.com/paper/2410.05454/) (2024)
- [GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning](https://mlsanity.com/paper/2410.23889/) (2024)
- [Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing](https://mlsanity.com/paper/2206.13336/) (2022)
- [Phase2vec: Dynamical systems embedding with a physics-informed convolutional network](https://mlsanity.com/paper/2212.03857/) (2022)
- [Spatio-Temporal Fluid Dynamics Modeling via Physical-Awareness and Parameter Diffusion Guidance](https://mlsanity.com/paper/2403.13850/) (2024)
- [Projected Neural Differential Equations for Learning Constrained Dynamics](https://mlsanity.com/paper/2410.23667/) (2024)
- [Discovering Physics Laws of Dynamical Systems via Invariant Function Learning](https://mlsanity.com/paper/2502.04495/) (2025)
- [Towards Foundational Models for Dynamical System Reconstruction: Hierarchical Meta-Learning via Mixture of Experts](https://mlsanity.com/paper/2502.05335/) (2025)
- [Towards Cross Domain Generalization of Hamiltonian Representation via Meta Learning](https://mlsanity.com/paper/2212.01168/) (2022)
- [SpReME: Sparse Regression for Multi-Environment Dynamic Systems](https://mlsanity.com/paper/2302.05942/) (2023)
- [Hybrid additive modeling with partial dependence for supervised regression and dynamical systems forecasting](https://mlsanity.com/paper/2307.02229/) (2023)
- [Reevaluating Meta-Learning Optimization Algorithms Through Contextual Self-Modulation](https://mlsanity.com/paper/2410.01655/) (2024)
- [Tailored Forecasting from Short Time Series via Meta-learning](https://mlsanity.com/paper/2501.16325/) (2025)
- [Environment Inference for Learning Generalizable Dynamical System](https://mlsanity.com/paper/2510.19784/) (2025)
- [Predicting Time Series of Networked Dynamical Systems without Knowing Topology](https://mlsanity.com/paper/2412.18734/) (2024)
- [Differentiable Physics-Neural Models enable Learning of Non-Markovian Closures for Accelerated Coarse-Grained Physics Simulations](https://mlsanity.com/paper/2511.21369/) (2025)
- [Unsupervised Learning of Hybrid Latent Dynamics: A Learn-to-Identify Framework](https://mlsanity.com/paper/2403.08194/) (2024)

## Abstract (arXiv)

When modeling dynamical systems from real-world data samples, the distribution of data often changes according to the environment in which they are captured, and the dynamics of the system itself vary from one environment to another. Generalizing across environments thus challenges the conventional frameworks. The classical settings suggest either considering data as i.i.d. and learning a single model to cover all situations or learning environment-specific models. Both are sub-optimal: the former disregards the discrepancies between environments leading to biased solutions, while the latter does not exploit their potential commonalities and is prone to scarcity problems. We propose LEADS, a novel framework that leverages the commonalities and discrepancies among known environments to improve model generalization. This is achieved with a tailored training formulation aiming at capturing common dynamics within a shared model while additional terms capture environment-specific dynamics. We ground our approach in theory, exhibiting a decrease in sample complexity with our approach and corroborate these results empirically, instantiating it for linear dynamics. Moreover, we concretize this framework for neural networks and evaluate it experimentally on representative families of nonlinear dynamics. We show that this new setting can exploit knowledge extracted from environment-dependent data and improves generalization for both known and novel environments. Code is available at https://github.com/yuan-yin/LEADS.