# A Neural PDE Solver with Temporal Stencil Modeling

arXiv: https://arxiv.org/abs/2302.08105 · HTML: https://mlsanity.com/paper/2302.08105/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Builds on (most cited first, 28 of 28)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [It's Raw! Audio Generation with State-Space Models](https://mlsanity.com/paper/2202.09729/) (2022)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [EikoNet: Solving the Eikonal equation with Deep Neural Networks](https://mlsanity.com/paper/2004.00361/) (2020)
- [Learning to correct spectral methods for simulating turbulent flows](https://mlsanity.com/paper/2207.00556/) (2022)
- [A posteriori learning for quasi-geostrophic turbulence parametrization](https://mlsanity.com/paper/2204.03911/) (2022)
- [Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems](https://mlsanity.com/paper/1904.05417/) (2019)
- [Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?](https://mlsanity.com/paper/2206.02016/) (2022)
- [A machine learning framework for data driven acceleration of computations of differential equations](https://mlsanity.com/paper/1807.09519/) (2018)

## Built on by (most cited first, 22 of 22)

- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Rolling Diffusion Models](https://mlsanity.com/paper/2402.09470/) (2024)
- [Recent Advances on Machine Learning for Computational Fluid Dynamics: A Survey](https://mlsanity.com/paper/2408.12171/) (2024)
- [BubbleML: A Multi-Physics Dataset and Benchmarks for Machine Learning](https://mlsanity.com/paper/2307.14623/) (2023)
- [Translation Equivariant Transformer Neural Processes](https://mlsanity.com/paper/2406.12409/) (2024)
- [P2C2Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics](https://mlsanity.com/paper/2411.00040/) (2024)
- [Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows](https://mlsanity.com/paper/2405.17211/) (2024)
- [Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates](https://mlsanity.com/paper/2412.13074/) (2024)
- [Geometric and Physical Constraints Synergistically Enhance Neural PDE Surrogates](https://mlsanity.com/paper/2506.05513/) (2025)
- [PeSANet: Physics-encoded Spectral Attention Network for Simulating PDE-Governed Complex Systems](https://mlsanity.com/paper/2505.01736/) (2025)
- [PAPM: A Physics-aware Proxy Model for Process Systems](https://mlsanity.com/paper/2407.05232/) (2024)
- [MultiPDENet: PDE-embedded Learning with Multi-time-stepping for Accelerated Flow Simulation](https://mlsanity.com/paper/2501.15987/) (2025)
- [INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers](https://mlsanity.com/paper/2511.12764/) (2025)
- [PIMRL: Physics-Informed Multi-Scale Recurrent Learning for Burst-Sampled Spatiotemporal Dynamics](https://mlsanity.com/paper/2503.10253/) (2025)
- [Spectral-inspired Operator Learning with Limited Data and Unknown Physics](https://mlsanity.com/paper/2505.21573/) (2025)
- [Learnable-Differentiable Finite Volume Solver for Accelerated Simulation of Flows](https://mlsanity.com/paper/2507.01975/) (2025)
- [Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems](https://mlsanity.com/paper/2602.11208/) (2026)
- [Stable Long-Horizon PDE Forecasting via Latent Structured Spectral Propagators](https://mlsanity.com/paper/2605.10154/) (2026)
- [Generalized Neural Operator for Parametric and Boundary-Value Problems](https://mlsanity.com/paper/2607.21932/) (2026)
- [OmniFluids: Physics Pre-trained Modeling of Fluid Dynamics](https://mlsanity.com/paper/2506.10862/) (2025)
- [Derived Fields Preserve Fine-Scale Detail in Budgeted Neural Simulators](https://mlsanity.com/paper/2603.29224/) (2026)
- [Hybrid Lagrangian-Eulerian Model for Lagrangian Fluid Simulation](https://mlsanity.com/paper/2608.01164/) (2026)

## Abstract (arXiv)

Numerical simulation of non-linear partial differential equations plays a crucial role in modeling physical science and engineering phenomena, such as weather, climate, and aerodynamics. Recent Machine Learning (ML) models trained on low-resolution spatio-temporal signals have shown new promises in capturing important dynamics in high-resolution signals, under the condition that the models can effectively recover the missing details. However, this study shows that significant information is often lost in the low-resolution down-sampled features. To address such issues, we propose a new approach, namely Temporal Stencil Modeling (TSM), which combines the strengths of advanced time-series sequence modeling (with the HiPPO features) and state-of-the-art neural PDE solvers (with learnable stencil modeling). TSM aims to recover the lost information from the PDE trajectories and can be regarded as a temporal generalization of classic finite volume methods such as WENO. Our experimental results show that TSM achieves the new state-of-the-art simulation accuracy for 2-D incompressible Navier-Stokes turbulent flows: it significantly outperforms the previously reported best results by 19.9% in terms of the highly-correlated duration time and reduces the inference latency into 80%. We also show a strong generalization ability of the proposed method to various out-of-distribution turbulent flow settings. Our code is available at "https://github.com/Edward-Sun/TSM-PDE".