# Message Passing Neural PDE Solvers

arXiv: https://arxiv.org/abs/2202.03376 · HTML: https://mlsanity.com/paper/2202.03376/

## Key facts

- Cited by 386 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 27 of 27)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Geometric and Physical Quantities Improve E(3) Equivariant Message Passing](https://mlsanity.com/paper/2110.02905/) (2021)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks](https://mlsanity.com/paper/1905.13166/) (2019)
- [Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction](https://mlsanity.com/paper/2007.04439/) (2020)
- [Learning to Optimize Multigrid PDE Solvers](https://mlsanity.com/paper/1902.10248/) (2019)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Probabilistic Numerics and Uncertainty in Computations](https://mlsanity.com/paper/1506.01326/) (2015)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [Combining Generative and Discriminative Models for Hybrid Inference](https://mlsanity.com/paper/1906.02547/) (2019)
- [Boundary Graph Neural Networks for 3D Simulations](https://mlsanity.com/paper/2106.11299/) (2021)
- [Probabilistic ODE Solvers with Runge-Kutta Means](https://mlsanity.com/paper/1406.2582/) (2014)

## Built on by (most cited first, 30 of 386)

- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction](https://mlsanity.com/paper/2211.14680/) (2022)
- [DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training](https://mlsanity.com/paper/2403.03542/) (2024)
- [Scalable Transformer for PDE Surrogate Modeling](https://mlsanity.com/paper/2305.17560/) (2023)
- [DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting](https://mlsanity.com/paper/2306.01984/) (2023)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [Operator Learning with Neural Fields: Tackling PDEs on General Geometries](https://mlsanity.com/paper/2306.07266/) (2023)
- [Lie Point Symmetry Data Augmentation for Neural PDE Solvers](https://mlsanity.com/paper/2202.07643/) (2022)
- [Continuous PDE Dynamics Forecasting with Implicit Neural Representations](https://mlsanity.com/paper/2209.14855/) (2022)
- [Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation](https://mlsanity.com/paper/2309.01745/) (2023)
- [A Survey on Graph Representation Learning Methods](https://mlsanity.com/paper/2204.01855/) (2022)
- [Neural Inverse Operators for Solving PDE Inverse Problems](https://mlsanity.com/paper/2301.11167/) (2023)
- [How does over-squashing affect the power of GNNs?](https://mlsanity.com/paper/2306.03589/) (2023)
- [From Zero to Turbulence: Generative Modeling for 3D Flow Simulation](https://mlsanity.com/paper/2306.01776/) (2023)
- [Training neural operators to preserve invariant measures of chaotic attractors](https://mlsanity.com/paper/2306.01187/) (2023)
- [Group Equivariant Fourier Neural Operators for Partial Differential Equations](https://mlsanity.com/paper/2306.05697/) (2023)
- [Mitigating spectral bias for the multiscale operator learning](https://mlsanity.com/paper/2210.10890/) (2022)
- [MgNO: Efficient Parameterization of Linear Operators via Multigrid](https://mlsanity.com/paper/2310.19809/) (2023)
- [Learning to Accelerate Partial Differential Equations via Latent Global Evolution](https://mlsanity.com/paper/2206.07681/) (2022)
- [AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields](https://mlsanity.com/paper/2406.02176/) (2024)
- [CodePDE: An Inference Framework for LLM-driven PDE Solver Generation](https://mlsanity.com/paper/2505.08783/) (2025)
- [Geometric Clifford Algebra Networks](https://mlsanity.com/paper/2302.06594/) (2023)

## Abstract (arXiv)

The numerical solution of partial differential equations (PDEs) is difficult, having led to a century of research so far. Recently, there have been pushes to build neural--numerical hybrid solvers, which piggy-backs the modern trend towards fully end-to-end learned systems. Most works so far can only generalize over a subset of properties to which a generic solver would be faced, including: resolution, topology, geometry, boundary conditions, domain discretization regularity, dimensionality, etc. In this work, we build a solver, satisfying these properties, where all the components are based on neural message passing, replacing all heuristically designed components in the computation graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. In order to encourage stability in training autoregressive models, we put forward a method that is based on the principle of zero-stability, posing stability as a domain adaptation problem. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.