# Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods

arXiv: https://arxiv.org/abs/2311.11798 · HTML: https://mlsanity.com/paper/2311.11798/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 89 references that are in the map.
- First posted on arXiv 20 Nov 2023.

## Builds on (most cited first, 29 of 29)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [In-Context Operator Learning with Data Prompts for Differential Equation Problems](https://mlsanity.com/paper/2304.07993/) (2023)
- [Neural Lander: Stable Drone Landing Control using Learned Dynamics](https://mlsanity.com/paper/1811.08027/) (2018)
- [Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems](https://mlsanity.com/paper/2203.15706/) (2022)
- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling](https://mlsanity.com/paper/2109.06407/) (2021)
- [Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks](https://mlsanity.com/paper/1808.03620/) (2018)
- [Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2109.00060/) (2021)
- [Neural Closure Models for Dynamical Systems](https://mlsanity.com/paper/2012.13869/) (2020)
- [Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2210.03008/) (2022)
- [Ensemble Kalman Methods: A Mean Field Perspective](https://mlsanity.com/paper/2209.11371/) (2022)
- [Variationally Mimetic Operator Networks](https://mlsanity.com/paper/2209.12871/) (2022)
- [PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformers](https://mlsanity.com/paper/2309.16816/) (2023)
- [Tikhonov Regularization Within Ensemble Kalman Inversion](https://mlsanity.com/paper/1901.10382/) (2019)
- [Gradient-free training of neural ODEs for system identification and control using ensemble Kalman inversion](https://mlsanity.com/paper/2307.07882/) (2023)
- [A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization](https://mlsanity.com/paper/2208.09104/) (2022)
- [CEBoosting: Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting](https://mlsanity.com/paper/2304.07863/) (2023)
- [Ensemble Kalman Methods With Constraints](https://mlsanity.com/paper/1901.05668/) (2019)
- [lp regularization for ensemble Kalman inversion](https://mlsanity.com/paper/2009.03470/) (2020)

## Built on by (most cited first, 14 of 14)

- [Data-Driven Stochastic Closure Modeling via Conditional Diffusion Model and Neural Operator](https://mlsanity.com/paper/2408.02965/) (2024)
- [Binned Spectral Power Loss for Improved Prediction of Chaotic Systems](https://mlsanity.com/paper/2502.00472/) (2025)
- [CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation](https://mlsanity.com/paper/2410.20072/) (2024)
- [CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation](https://mlsanity.com/paper/2404.06749/) (2024)
- [Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network](https://mlsanity.com/paper/2507.08749/) (2025)
- [Active Learning of Model Discrepancy with Bayesian Experimental Design](https://mlsanity.com/paper/2502.05372/) (2025)
- [Stochastic and Non-local Closure Modeling for Nonlinear Dynamical Systems via Latent Score-based Generative Models](https://mlsanity.com/paper/2506.20771/) (2025)
- [Implicit factorized transformer approach to fast prediction of turbulent channel flows](https://mlsanity.com/paper/2412.18840/) (2024)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [An Inverse Scattering Inspired Fourier Neural Operator for Time-Dependent PDE Learning](https://mlsanity.com/paper/2512.19439/) (2025)
- [Weak-Form Evolutionary Kolmogorov-Arnold Networks for Solving Partial Differential Equations](https://mlsanity.com/paper/2602.18515/) (2026)
- [Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance](https://mlsanity.com/paper/2601.16425/) (2026)
- [TF-SNO: Time-Frequency Gated Spectral Neural Operators for Learning Non-Stationary Partial Differential Equations](https://mlsanity.com/paper/2606.21189/) (2026)
- [Geometry-aware Incremental Neural Operator for Long-Horizon PDE prediction](https://mlsanity.com/paper/2608.11237/) (2026)

## Abstract (arXiv)

Data-driven modeling techniques have been explored in the spatial-temporal modeling of complex dynamical systems for many engineering applications. However, a systematic approach is still lacking to leverage the information from different types of data, e.g., with different spatial and temporal resolutions, and the combined use of short-term trajectories and long-term statistics. In this work, we build on the recent progress of neural operator and present a data-driven modeling framework called neural dynamical operator that is continuous in both space and time. A key feature of the neural dynamical operator is the resolution-invariance with respect to both spatial and temporal discretizations, without demanding abundant training data in different temporal resolutions. To improve the long-term performance of the calibrated model, we further propose a hybrid optimization scheme that leverages both gradient-based and derivative-free optimization methods and efficiently trains on both short-term time series and long-term statistics. We investigate the performance of the neural dynamical operator with three numerical examples, including the viscous Burgers' equation, the Navier-Stokes equations, and the Kuramoto-Sivashinsky equation. The results confirm the resolution-invariance of the proposed modeling framework and also demonstrate stable long-term simulations with only short-term time series data. In addition, we show that the proposed model can better predict long-term statistics via the hybrid optimization scheme with a combined use of short-term and long-term data.