# Meta-Learning Dynamics Forecasting Using Task Inference

arXiv: https://arxiv.org/abs/2102.10271 · HTML: https://mlsanity.com/paper/2102.10271/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 94 references that are in the map.
- First posted on arXiv 20 Feb 2021.

## Builds on (most cited first, 30 of 60)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Deep multi-scale video prediction beyond mean square error](https://mlsanity.com/paper/1511.05440/) (2015)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [Unsupervised Learning for Physical Interaction through Video Prediction](https://mlsanity.com/paper/1605.07157/) (2016)
- [PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing](https://mlsanity.com/paper/1805.04409/) (2018)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction](https://mlsanity.com/paper/2003.01460/) (2020)

## Built on by (most cited first, 30 of 38)

- [DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting](https://mlsanity.com/paper/2306.01984/) (2023)
- [On the Connection Between MPNN and Graph Transformer](https://mlsanity.com/paper/2301.11956/) (2023)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [Generalizing to New Physical Systems via Context-Informed Dynamics Model](https://mlsanity.com/paper/2202.01889/) (2022)
- [Training neural operators to preserve invariant measures of chaotic attractors](https://mlsanity.com/paper/2306.01187/) (2023)
- [Physics-Guided Deep Learning for Dynamical Systems: A Survey](https://mlsanity.com/paper/2107.01272/) (2021)
- [Multi-Task Imitation Learning for Linear Dynamical Systems](https://mlsanity.com/paper/2212.00186/) (2022)
- [Physics Informed Token Transformer for Solving Partial Differential Equations](https://mlsanity.com/paper/2305.08757/) (2023)
- [DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic prediction](https://mlsanity.com/paper/2504.19496/) (2025)
- [Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting](https://mlsanity.com/paper/2206.09450/) (2022)
- [Deciphering and integrating invariants for neural operator learning with various physical mechanisms](https://mlsanity.com/paper/2311.14361/) (2023)
- [Masked Autoencoders are PDE Learners](https://mlsanity.com/paper/2403.17728/) (2024)
- [SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations](https://mlsanity.com/paper/2403.19507/) (2024)
- [Zero-shot Imputation with Foundation Inference Models for Dynamical Systems](https://mlsanity.com/paper/2402.07594/) (2024)
- [Interpretable Meta-Learning of Physical Systems](https://mlsanity.com/paper/2312.00477/) (2023)
- [MetaNO: How to Transfer Your Knowledge on Learning Hidden Physics](https://mlsanity.com/paper/2301.12095/) (2023)
- [Test-time Generalization for Physics through Neural Operator Splitting](https://mlsanity.com/paper/2602.00884/) (2026)
- [Neural Context Flows for Meta-Learning of Dynamical Systems](https://mlsanity.com/paper/2405.02154/) (2024)
- [Meta-Dynamical State Space Models for Integrative Neural Data Analysis](https://mlsanity.com/paper/2410.05454/) (2024)
- [GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning](https://mlsanity.com/paper/2410.23889/) (2024)
- [Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing](https://mlsanity.com/paper/2206.13336/) (2022)
- [MetaPhysiCa: OOD Robustness in Physics-informed Machine Learning](https://mlsanity.com/paper/2303.03181/) (2023)
- [Discovering Physics Laws of Dynamical Systems via Invariant Function Learning](https://mlsanity.com/paper/2502.04495/) (2025)
- [Towards Foundational Models for Dynamical System Reconstruction: Hierarchical Meta-Learning via Mixture of Experts](https://mlsanity.com/paper/2502.05335/) (2025)
- [Towards Cross Domain Generalization of Hamiltonian Representation via Meta Learning](https://mlsanity.com/paper/2212.01168/) (2022)
- [Gaussian Process Latent Variable Modeling for Few-shot Time Series Forecasting](https://mlsanity.com/paper/2212.10306/) (2022)
- [Inductive biases in deep learning models for weather prediction](https://mlsanity.com/paper/2304.04664/) (2023)
- [The Power of Next-Frame Prediction for Learning Physical Laws](https://mlsanity.com/paper/2405.17450/) (2024)
- [Learning to learn ecosystems from limited data -- a meta-learning approach](https://mlsanity.com/paper/2410.07368/) (2024)
- [Tailored Forecasting from Short Time Series via Meta-learning](https://mlsanity.com/paper/2501.16325/) (2025)

## Abstract (arXiv)

Current deep learning models for dynamics forecasting struggle with generalization. They can only forecast in a specific domain and fail when applied to systems with different parameters, external forces, or boundary conditions. We propose a model-based meta-learning method called DyAd which can generalize across heterogeneous domains by partitioning them into different tasks. DyAd has two parts: an encoder which infers the time-invariant hidden features of the task with weak supervision, and a forecaster which learns the shared dynamics of the entire domain. The encoder adapts and controls the forecaster during inference using adaptive instance normalization and adaptive padding. Theoretically, we prove that the generalization error of such procedure is related to the task relatedness in the source domain, as well as the domain differences between source and target. Experimentally, we demonstrate that our model outperforms state-of-the-art approaches on both turbulent flow and real-world ocean data forecasting tasks.