# Implicit Neural Differential Model for Spatiotemporal Dynamics

arXiv: https://arxiv.org/abs/2504.02260 · HTML: https://mlsanity.com/paper/2504.02260/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Physics-informed learning of governing equations from scarce data](https://mlsanity.com/paper/2005.03448/) (2020)
- [Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction](https://mlsanity.com/paper/2007.04439/) (2020)
- [Enhancing Computational Fluid Dynamics with Machine Learning](https://mlsanity.com/paper/2110.02085/) (2021)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [A Differentiable Programming System to Bridge Machine Learning and Scientific Computing](https://mlsanity.com/paper/1907.07587/) (2019)
- [Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics](https://mlsanity.com/paper/2205.03990/) (2022)
- [The Elements of Differentiable Programming](https://mlsanity.com/paper/2403.14606/) (2024)
- [Learned Turbulence Modelling with Differentiable Fluid Solvers: Physics-based Loss-functions and Optimisation Horizons](https://mlsanity.com/paper/2202.06988/) (2022)
- [A Review of Differentiable Simulators](https://mlsanity.com/paper/2407.05560/) (2024)
- [Differentiability in Unrolled Training of Neural Physics Simulators on Transient Dynamics](https://mlsanity.com/paper/2402.12971/) (2024)
- [Differentiable hybrid neural modeling for fluid-structure interaction](https://mlsanity.com/paper/2303.12971/) (2023)
- [Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertainty](https://mlsanity.com/paper/2210.08095/) (2022)
- [AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models](https://mlsanity.com/paper/2307.10711/) (2023)
- [DiffHybrid-UQ: Uncertainty Quantification for Differentiable Hybrid Neural Modeling](https://mlsanity.com/paper/2401.00161/) (2023)
- [Prediction of Reynolds Stresses in High-Mach-Number Turbulent Boundary Layers using Physics-Informed Machine Learning](https://mlsanity.com/paper/1808.07752/) (2018)
- [Probabilistic Physics-integrated Neural Differentiable Modeling for Isothermal Chemical Vapor Infiltration Process](https://mlsanity.com/paper/2311.07798/) (2023)

## Built on by (most cited first, 1 of 1)

- [Differentiable Autoencoding Neural Operator for Interpretable and Integrable Latent Space Modeling](https://mlsanity.com/paper/2510.00233/) (2025)

## Abstract (arXiv)

Hybrid neural-physics modeling frameworks through differentiable programming have emerged as powerful tools in scientific machine learning, enabling the integration of known physics with data-driven learning to improve prediction accuracy and generalizability. However, most existing hybrid frameworks rely on explicit recurrent formulations, which suffer from numerical instability and error accumulation during long-horizon forecasting. In this work, we introduce Im-PiNDiff, a novel implicit physics-integrated neural differentiable solver for stable and accurate modeling of spatiotemporal dynamics. Inspired by deep equilibrium models, Im-PiNDiff advances the state using implicit fixed-point layers, enabling robust long-term simulation while remaining fully end-to-end differentiable. To enable scalable training, we introduce a hybrid gradient propagation strategy that integrates adjoint-state methods with reverse-mode automatic differentiation. This approach eliminates the need to store intermediate solver states and decouples memory complexity from the number of solver iterations, significantly reducing training overhead. We further incorporate checkpointing techniques to manage memory in long-horizon rollouts. Numerical experiments on various spatiotemporal PDE systems, including advection-diffusion processes, Burgers' dynamics, and multi-physics chemical vapor infiltration processes, demonstrate that Im-PiNDiff achieves superior predictive performance, enhanced numerical stability, and substantial reductions in memory and runtime cost relative to explicit and naive implicit baselines. This work provides a principled, efficient, and scalable framework for hybrid neural-physics modeling.