mlsanityData to 7 Oct 2026

Neural Context Flows for Meta-Learning of Dynamical Systems

arXiv 2405.02154 ↗First posted 3 May 2024cs.LG math.DScited by 9

Not on the map. Why?

Abstract (arXiv)

Neural Ordinary Differential Equations (NODEs) often struggle to adapt to new dynamic behaviors caused by parameter changes in the underlying physical system, even when these dynamics are similar to previously observed behaviors. This problem becomes more challenging when the changing parameters are unobserved, meaning their value or influence cannot be directly measured when collecting data. To address this issue, we introduce Neural Context Flow (NCF), a robust and interpretable Meta-Learning framework that includes uncertainty estimation. NCF uses Taylor expansion to enable contextual self-modulation, allowing context vectors to influence dynamics from other domains while also modulating themselves. After establishing theoretical guarantees, we empirically test NCF and compare it to related adaptation methods. Our results show that NCF achieves state-of-the-art Out-of-Distribution performance on 5 out of 6 linear and non-linear benchmark problems. Through extensive experiments, we explore the flexible model architecture of NCF and the encoded representations within the learned context vectors. Our findings highlight the potential implications of NCF for foundational models in the physical sciences, offering a promising approach to improving the adaptability and generalization of NODEs in various scientific applications. Our code is openly available at https://github.com/ddrous/ncflow.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 38 of its 78 references that are in the map. First posted on arXiv 3 May 2024.

Builds on

38 of 78 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  6. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  7. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  8. 2017Learning multiple visual domains with residual adapterscited by 553
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  10. 2018Conditional Neural Processescited by 421
  11. 2019Augmented Neural ODEscited by 396
  12. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
26 more
  1. 2020Universal Differential Equations for Scientific Machine Learningcited by 345
  2. 2017Stable Architectures for Deep Neural Networkscited by 340
  3. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 288
  4. 2020AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradientscited by 282
  5. 2022On Neural Differential Equationscited by 264
  6. 2023Neural General Circulation Models for Weather and Climatecited by 232
  7. 2018Efficient parametrization of multi-domain deep neural networkscited by 168
  8. 2017Searching for Activation Functionscited by 158
  9. 2020Dissecting Neural ODEscited by 150
  10. 2021Equinox: neural networks in JAX via callable PyTrees and filtered transformationscited by 143
  11. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 128
  12. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  13. 2020Augmenting Physical Models with Deep Networks for Complex Dynamics Forecastingcited by 93
  14. 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 90
  15. 2021Out of Distribution Generalization in Machine Learningcited by 59
  16. 2022Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practicescited by 54
  17. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  18. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 49
  19. 2020Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problemscited by 45
  20. 2021LEADS: Learning Dynamical Systems that Generalize Across Environmentscited by 39
  21. 2021Meta-Learning Dynamics Forecasting Using Task Inferencecited by 37
  22. 2023Learning Neural PDE Solvers with Parameter-Guided Channel Attentioncited by 37
  23. 2020ChemNODE: A Neural Ordinary Differential Equations Approach for Chemical Kinetics Solverscited by 26
  24. 2019Machine Learning and Big Scientific Datacited by 16
  25. 2023Interpretable Meta-Learning of Physical Systemscited by 11
  26. 2023A Comparison of Mesh-Free Differentiable Programming and Data-Driven Strategies for Optimal Control under PDE Constraintscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 4.

9 papers.

  1. 2024Zebra: In-Context Generative Pretraining for Solving Parametric PDEscited by 14
  2. 2026Test-time Generalization for Physics through Neural Operator Splittingcited by 8
  3. 2025Towards Foundational Models for Dynamical System Reconstruction: Hierarchical Meta-Learning via Mixture of Expertscited by 5
  4. 2025Weight-Space Linear Recurrent Neural Networkscited by 5
  5. 2025Position: Biology is the Challenge Physics-Informed ML Needs to Evolvecited by 4
  6. 2024Reevaluating Meta-Learning Optimization Algorithms Through Contextual Self-Modulationcited by 2
  7. 2026Out-of-Support Generalisation via Weight-Space Sequence Modellingcited by 1
  8. 2026Render, Don't Decode: Weight-Space World Models with Latent Structural Disentanglementcited by 0
  9. 2026Generalized Neural Operator for Parametric and Boundary-Value Problemscited by 0