# Long-time integration of parametric evolution equations with physics-informed DeepONets

arXiv: https://arxiv.org/abs/2106.05384 · HTML: https://mlsanity.com/paper/2106.05384/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 63 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Variational Physics-Informed Neural Networks For Solving Partial Differential Equations](https://mlsanity.com/paper/1912.00873/) (2019)
- [Understanding and mitigating gradient pathologies in physics-informed neural networks](https://mlsanity.com/paper/2001.04536/) (2020)
- [Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks](https://mlsanity.com/paper/1906.05747/) (2019)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications](https://mlsanity.com/paper/1908.10407/) (2019)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks](https://mlsanity.com/paper/1905.04817/) (2019)
- [NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework](https://mlsanity.com/paper/2012.07938/) (2020)
- [Evolutional Deep Neural Network](https://mlsanity.com/paper/2103.09959/) (2021)
- [Inferring solutions of differential equations using noisy multi-fidelity data](https://mlsanity.com/paper/1607.04805/) (2016)
- [Deep Learning of Turbulent Scalar Mixing](https://mlsanity.com/paper/1811.07095/) (2018)

## Built on by (most cited first, 30 of 89)

- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Improved architectures and training algorithms for deep operator networks](https://mlsanity.com/paper/2110.01654/) (2021)
- [Continuous PDE Dynamics Forecasting with Implicit Neural Representations](https://mlsanity.com/paper/2209.14855/) (2022)
- [Generic bounds on the approximation error for physics-informed (and) operator learning](https://mlsanity.com/paper/2205.11393/) (2022)
- [Error estimates for physics informed neural networks approximating the Navier-Stokes equations](https://mlsanity.com/paper/2203.09346/) (2022)
- [Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems](https://mlsanity.com/paper/2204.09157/) (2022)
- [Physics-Guided Deep Learning for Dynamical Systems: A Survey](https://mlsanity.com/paper/2107.01272/) (2021)
- [Learning to Solve PDE-constrained Inverse Problems with Graph Networks](https://mlsanity.com/paper/2206.00711/) (2022)
- [SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability](https://mlsanity.com/paper/2204.12670/) (2022)
- [Implicit Neural Spatial Representations for Time-dependent PDEs](https://mlsanity.com/paper/2210.00124/) (2022)
- [A Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEs](https://mlsanity.com/paper/2110.13361/) (2021)
- [Neural Operator Learning for Long-Time Integration in Dynamical Systems with Recurrent Neural Networks](https://mlsanity.com/paper/2303.02243/) (2023)
- [wPINNs: Weak Physics informed neural networks for approximating entropy solutions of hyperbolic conservation laws](https://mlsanity.com/paper/2207.08483/) (2022)
- [Transfer Learning Enhanced DeepONet for Long-Time Prediction of Evolution Equations](https://mlsanity.com/paper/2212.04663/) (2022)
- [Bi-fidelity Modeling of Uncertain and Partially Unknown Systems using DeepONets](https://mlsanity.com/paper/2204.00997/) (2022)
- [Fast PDE-constrained optimization via self-supervised operator learning](https://mlsanity.com/paper/2110.13297/) (2021)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [Improving physics-informed neural networks with meta-learned optimization](https://mlsanity.com/paper/2303.07127/) (2023)
- [Data-driven anisotropic finite viscoelasticity using neural ordinary differential equations](https://mlsanity.com/paper/2302.03598/) (2023)
- [TI-DeepONet: Learnable Time Integration for Stable Long-Term Extrapolation](https://mlsanity.com/paper/2505.17341/) (2025)
- [Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertainty](https://mlsanity.com/paper/2210.08095/) (2022)
- [On Approximating the Dynamic Response of Synchronous Generators via Operator Learning: A Step Towards Building Deep Operator-based Power Grid Simulators](https://mlsanity.com/paper/2301.12538/) (2023)
- [A multifidelity approach to continual learning for physical systems](https://mlsanity.com/paper/2304.03894/) (2023)
- [Exactly conservative physics-informed neural networks and deep operator networks for dynamical systems](https://mlsanity.com/paper/2311.14131/) (2023)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [NeuFENet: Neural Finite Element Solutions with Theoretical Bounds for Parametric PDEs](https://mlsanity.com/paper/2110.01601/) (2021)

## Abstract (arXiv)

Ordinary and partial differential equations (ODEs/PDEs) play a paramount role in analyzing and simulating complex dynamic processes across all corners of science and engineering. In recent years machine learning tools are aspiring to introduce new effective ways of simulating PDEs, however existing approaches are not able to reliably return stable and accurate predictions across long temporal horizons. We aim to address this challenge by introducing an effective framework for learning infinite-dimensional operators that map random initial conditions to associated PDE solutions within a short time interval. Such latent operators can be parametrized by deep neural networks that are trained in an entirely self-supervised manner without requiring any paired input-output observations. Global long-time predictions across a range of initial conditions can be then obtained by iteratively evaluating the trained model using each prediction as the initial condition for the next evaluation step. This introduces a new approach to temporal domain decomposition that is shown to be effective in performing accurate long-time simulations for a wide range of parametric ODE and PDE systems, from wave propagation, to reaction-diffusion dynamics and stiff chemical kinetics, all at a fraction of the computational cost needed by classical numerical solvers.