mlsanityData to 7 Oct 2026

Predicting Physics in Mesh-reduced Space with Temporal Attention

arXiv 2201.09113 ↗First posted 22 Jan 2022cs.LGcited by 82

Not on the map. Why?

Abstract (arXiv)

Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes. However, due to their short temporal attention span, these models suffer from error accumulation and drift. In this paper, we propose a new method that captures long-term dependencies through a transformer-style temporal attention model. We introduce an encoder-decoder structure to summarize features and create a compact mesh representation of the system state, to allow the temporal model to operate on a low-dimensional mesh representations in a memory efficient manner. Our method outperforms a competitive GNN baseline on several complex fluid dynamics prediction tasks, from sonic shocks to vascular flow. We demonstrate stable rollouts without the need for training noise and show perfectly phase-stable predictions even for very long sequences. More broadly, we believe our approach paves the way to bringing the benefits of attention-based sequence models to solving high-dimensional complex physics tasks.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 30 of its 55 references that are in the map. First posted on arXiv 22 Jan 2022.

Builds on

30 of 55 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  4. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  5. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  6. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  7. 2018Image Transformercited by 770
  8. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  9. 2019Graph U-Netscited by 489
  10. 2021Machine learning accelerated computational fluid dynamicscited by 461
  11. 2018Graph networks as learnable physics engines for inference and controlcited by 242
  12. 2018Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluidscited by 219
18 more
  1. 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 203
  2. 2020Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solverscited by 201
  3. 2020PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domaincited by 173
  4. 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
  5. 2018Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flowscited by 147
  6. 2019Flow-based generative models for Markov chain Monte Carlo in lattice field theorycited by 147
  7. 2019Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networkscited by 144
  8. 2020Transformers for Modeling Physical Systemscited by 105
  9. 2020Equivariant flow-based sampling for lattice gauge theorycited by 102
  10. 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
  11. 2021POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decompositioncited by 79
  12. 2020Physics-Informed Multi-LSTM Networks for Metamodeling of Nonlinear Structurescited by 52
  13. 2019Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamicscited by 45
  14. 2019Graph Element Networks: adaptive, structured computation and memorycited by 41
  15. 2020Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flowscited by 36
  16. 2021Order Matters: Probabilistic Modeling of Node Sequence for Graph Generationcited by 32
  17. 2017Liquid Splash Modeling with Neural Networkscited by 16
  18. 2021Conditionally Parameterized, Discretization-Aware Neural Networks for Mesh-Based Modeling of Physical Systemscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 20; 2024: 20; 2025: 19; 2026: 15.

82 papers.

Sort:

By area:

  1. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  2. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  3. 2023DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecastingcited by 105
  4. 2022MultiScale MeshGraphNetscited by 94
  5. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  6. 2024Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equationscited by 87
  7. 2023Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulationcited by 65
  8. 2023Eagle: Large-Scale Learning of Turbulent Fluid Dynamics with Mesh Transformerscited by 54
  9. 2023A graph convolutional autoencoder approach to model order reduction for parametrized PDEscited by 44
  10. 2022Efficient Learning of Mesh-Based Physical Simulation with BSMS-GNNcited by 43
  11. 2025Generative Latent Neural PDE Solver using Flow Matchingcited by 28
  12. 2024Recent Advances on Machine Learning for Computational Fluid Dynamics: A Surveycited by 27
38 more
  1. 2024HAMLET: Graph Transformer Neural Operator for Partial Differential Equationscited by 24
  2. 2023Physics Informed Token Transformer for Solving Partial Differential Equationscited by 21
  3. 2023Learning Controllable Adaptive Simulation for Multi-resolution Physicscited by 20
  4. 2025Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulationcited by 19
  5. 2023Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformercited by 18
  6. 2024PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systemscited by 18
  7. 2022Latent Neural ODEs with Sparse Bayesian Multiple Shootingcited by 17
  8. 2023Swarm Reinforcement Learning For Adaptive Mesh Refinementcited by 16
  9. 2024Generative Learning of the Solution of Parametric Partial Differential Equations Using Guided Diffusion Models and Virtual Observationscited by 16
  10. 2023Grounding Graph Network Simulators using Physical Sensor Observationscited by 15
  11. 2023Evolve Smoothly, Fit Consistently: Learning Smooth Latent Dynamics For Advection-Dominated Systemscited by 14
  12. 2023Multi-scale Time-stepping of Partial Differential Equations with Transformerscited by 14
  13. 2022Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertaintycited by 13
  14. 2025Machine learning for modelling unstructured grid data in computational physics: a reviewcited by 12
  15. 2025Mesh-Informed Neural Operator : A Transformer Generative Approachcited by 11
  16. 2025Diffusion-Based Hierarchical Graph Neural Networks for Simulating Nonlinear Solid Mechanicscited by 10
  17. 2025Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Modelscited by 8
  18. 2023Efficient Graph Field Integrators Meet Point Cloudscited by 7
  19. 2023Graph Neural Network for spatiotemporal data: methods and applicationscited by 6
  20. 2024Space and Time Continuous Physics Simulation From Partial Observationscited by 6
  21. 2024Generative Learning for Forecasting the Dynamics of Complex Systemscited by 6
  22. 2025Improving Long-Range Interactions in Graph Neural Simulators via Hamiltonian Dynamicscited by 6
  23. 2024Latent Neural PDE Solver: a reduced-order modelling framework for partial differential equationscited by 5
  24. 2024Towards Universal Mesh Movement Networkscited by 4
  25. 2024EvoMesh: Adaptive Physical Simulation with Hierarchical Graph Evolutionscited by 4
  26. 2025FD-Bench: A Modular and Fair Benchmark for Data-driven Fluid Simulationcited by 4
  27. 2023Learning CO2 plume migration in faulted reservoirs with Graph Neural Networkscited by 3
  28. 2023Learning Space-Time Continuous Neural PDEs from Partially Observed Statescited by 3
  29. 2023On the Relationships between Graph Neural Networks for the Simulation of Physical Systems and Classical Numerical Methodscited by 2
  30. 2024LatticeGraphNet: A two-scale graph neural operator for simulating lattice structurescited by 2
  31. 2024Learning Spatiotemporal Dynamical Systems from Point Process Observationscited by 2
  32. 2024Learning Physical Simulation with Message Passing Transformercited by 2
  33. 2024Learning Lagrangian Interaction Dynamics with Sampling-Based Model Order Reductioncited by 2
  34. 2025Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systemscited by 2
  35. 2025FIGNN: Feature-Specific Interpretability for Graph Neural Network Surrogate Modelscited by 2
  36. 2025GITO: Graph-Informed Transformer Operator for Learning Complex Partial Differential Equationscited by 2
  37. 2025Point-wise Diffusion Models for Physical Systems with Shape Variations: Application to Spatio-temporal and Large-scale systemcited by 2
  38. 2025M4GN: Mesh-based Multi-segment Hierarchical Graph Network for Dynamic Simulationscited by 2

50 most cited of 82 All 82 (JSON)