# Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks

arXiv: https://arxiv.org/abs/2203.08852 · HTML: https://mlsanity.com/paper/2203.08852/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Graph Neural Network for Traffic Forecasting: A Survey](https://mlsanity.com/paper/2101.11174/) (2021)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [Graph Neural Ordinary Differential Equations](https://mlsanity.com/paper/1911.07532/) (2019)
- [Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction](https://mlsanity.com/paper/2007.04439/) (2020)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [Examining COVID-19 Forecasting using Spatio-Temporal Graph Neural Networks](https://mlsanity.com/paper/2007.03113/) (2020)
- [Learning continuous-time PDEs from sparse data with graph neural networks](https://mlsanity.com/paper/2006.08956/) (2020)
- [Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows](https://mlsanity.com/paper/2005.13420/) (2020)
- [Learning Dynamical Systems from Partial Observations](https://mlsanity.com/paper/1902.11136/) (2019)
- [Hybrid FEM-NN models: Combining artificial neural networks with the finite element method](https://mlsanity.com/paper/2101.00962/) (2021)
- [Machine Learning for Spatiotemporal Sequence Forecasting: A Survey](https://mlsanity.com/paper/1808.06865/) (2018)
- [Incrementally Improving Graph WaveNet Performance on Traffic Prediction](https://mlsanity.com/paper/1912.07390/) (2019)
- [Learning Similarity Metrics for Numerical Simulations](https://mlsanity.com/paper/2002.07863/) (2020)

## Built on by (most cited first, 30 of 34)

- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [Eagle: Large-Scale Learning of Turbulent Fluid Dynamics with Mesh Transformers](https://mlsanity.com/paper/2302.10803/) (2023)
- [From Zero to Turbulence: Generative Modeling for 3D Flow Simulation](https://mlsanity.com/paper/2306.01776/) (2023)
- [Koopman neural operator as a mesh-free solver of non-linear partial differential equations](https://mlsanity.com/paper/2301.10022/) (2023)
- [Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks in Scientific Computing](https://mlsanity.com/paper/2211.07377/) (2022)
- [BENO: Boundary-embedded Neural Operators for Elliptic PDEs](https://mlsanity.com/paper/2401.09323/) (2024)
- [Transform Once: Efficient Operator Learning in Frequency Domain](https://mlsanity.com/paper/2211.14453/) (2022)
- [Inducing Point Operator Transformer: A Flexible and Scalable Architecture for Solving PDEs](https://mlsanity.com/paper/2312.10975/) (2023)
- [Latent Neural ODEs with Sparse Bayesian Multiple Shooting](https://mlsanity.com/paper/2210.03466/) (2022)
- [A Finite Element-Inspired Hypergraph Neural Network: Application to Fluid Dynamics Simulations](https://mlsanity.com/paper/2212.14545/) (2022)
- [Knowledge-guided Machine Learning: Current Trends and Future Prospects](https://mlsanity.com/paper/2403.15989/) (2024)
- [SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations](https://mlsanity.com/paper/2403.19507/) (2024)
- [Neural SPH: Improved Neural Modeling of Lagrangian Fluid Dynamics](https://mlsanity.com/paper/2402.06275/) (2024)
- [torchode: A Parallel ODE Solver for PyTorch](https://mlsanity.com/paper/2210.12375/) (2022)
- [Learning time-dependent PDE via graph neural networks and deep operator network for robust accuracy on irregular grids](https://mlsanity.com/paper/2402.08187/) (2024)
- [Improving Long-Range Interactions in Graph Neural Simulators via Hamiltonian Dynamics](https://mlsanity.com/paper/2511.08185/) (2025)
- [Spatio-Temporal Fluid Dynamics Modeling via Physical-Awareness and Parameter Diffusion Guidance](https://mlsanity.com/paper/2403.13850/) (2024)
- [Graph Network Surrogate Model for Subsurface Flow Optimization](https://mlsanity.com/paper/2312.08625/) (2023)
- [JAX-SPH: A Differentiable Smoothed Particle Hydrodynamics Framework](https://mlsanity.com/paper/2403.04750/) (2024)
- [Unfolding Time: Generative Modeling for Turbulent Flows in 4D](https://mlsanity.com/paper/2406.11390/) (2024)
- [Up-sampling-only and Adaptive Mesh-based GNN for Simulating Physical Systems](https://mlsanity.com/paper/2409.04740/) (2024)
- [Learning PDE Solution Operator for Continuous Modeling of Time-Series](https://mlsanity.com/paper/2302.00854/) (2023)
- [On the Relationships between Graph Neural Networks for the Simulation of Physical Systems and Classical Numerical Methods](https://mlsanity.com/paper/2304.00146/) (2023)
- [Learning Space-Time Continuous Neural PDEs from Partially Observed States](https://mlsanity.com/paper/2307.04110/) (2023)
- [G-Adaptivity: optimised graph-based mesh relocation for finite element methods](https://mlsanity.com/paper/2407.04516/) (2024)
- [Learning Efficient Surrogate Dynamic Models with Graph Spline Networks](https://mlsanity.com/paper/2310.16397/) (2023)
- [Learning Spatiotemporal Dynamical Systems from Point Process Observations](https://mlsanity.com/paper/2406.00368/) (2024)
- [FA-INR: Adaptive Implicit Neural Representations for Interpretable Exploration of Simulation Ensembles](https://mlsanity.com/paper/2506.06858/) (2025)
- [Scientific Computing Algorithms to Learn Enhanced Scalable Surrogates for Mesh Physics](https://mlsanity.com/paper/2304.00338/) (2023)

## Abstract (arXiv)

We propose a new method for spatio-temporal forecasting on arbitrarily distributed points. Assuming that the observed system follows an unknown partial differential equation, we derive a continuous-time model for the dynamics of the data via the finite element method. The resulting graph neural network estimates the instantaneous effects of the unknown dynamics on each cell in a meshing of the spatial domain. Our model can incorporate prior knowledge via assumptions on the form of the unknown PDE, which induce a structural bias towards learning specific processes. Through this mechanism, we derive a transport variant of our model from the convection equation and show that it improves the transfer performance to higher-resolution meshes on sea surface temperature and gas flow forecasting against baseline models representing a selection of spatio-temporal forecasting methods. A qualitative analysis shows that our model disentangles the data dynamics into their constituent parts, which makes it uniquely interpretable.