# Eagle: Large-Scale Learning of Turbulent Fluid Dynamics with Mesh Transformers

arXiv: https://arxiv.org/abs/2302.10803 · HTML: https://mlsanity.com/paper/2302.10803/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 56 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Skillful Precipitation Nowcasting using Deep Generative Models of Radar](https://mlsanity.com/paper/2104.00954/) (2021)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction](https://mlsanity.com/paper/2003.01460/) (2020)
- [Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows](https://mlsanity.com/paper/1810.08217/) (2018)
- [PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs](https://mlsanity.com/paper/2106.14103/) (2021)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [Predicting Physics in Mesh-reduced Space with Temporal Attention](https://mlsanity.com/paper/2201.09113/) (2022)
- [tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow](https://mlsanity.com/paper/1801.09710/) (2018)
- [A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries](https://mlsanity.com/paper/2010.09469/) (2020)
- [Physical Design using Differentiable Learned Simulators](https://mlsanity.com/paper/2202.00728/) (2022)
- [Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks](https://mlsanity.com/paper/2203.08852/) (2022)
- [Model Predictive Contouring Control for Time-Optimal Quadrotor Flight](https://mlsanity.com/paper/2108.13205/) (2021)
- [How Transferable are Reasoning Patterns in VQA?](https://mlsanity.com/paper/2104.03656/) (2021)
- [Latent Space Subdivision: Stable and Controllable Time Predictions for Fluid Flow](https://mlsanity.com/paper/2003.08723/) (2020)
- [Learning Similarity Metrics for Numerical Simulations](https://mlsanity.com/paper/2002.07863/) (2020)
- [Reconstructing High-resolution Turbulent Flows Using Physics-Guided Neural Networks](https://mlsanity.com/paper/2109.03327/) (2021)

## Built on by (most cited first, 30 of 57)

- [The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning](https://mlsanity.com/paper/2412.00568/) (2024)
- [Scalable Transformer for PDE Surrogate Modeling](https://mlsanity.com/paper/2305.17560/) (2023)
- [DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting](https://mlsanity.com/paper/2306.01984/) (2023)
- [From Zero to Turbulence: Generative Modeling for 3D Flow Simulation](https://mlsanity.com/paper/2306.01776/) (2023)
- [Generative Latent Neural PDE Solver using Flow Matching](https://mlsanity.com/paper/2503.22600/) (2025)
- [HAMLET: Graph Transformer Neural Operator for Partial Differential Equations](https://mlsanity.com/paper/2402.03541/) (2024)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems](https://mlsanity.com/paper/2502.17019/) (2025)
- [Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformer](https://mlsanity.com/paper/2312.12467/) (2023)
- [Multi-scale Time-stepping of Partial Differential Equations with Transformers](https://mlsanity.com/paper/2311.02225/) (2023)
- [P2C2Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics](https://mlsanity.com/paper/2411.00040/) (2024)
- [LaB-GATr: geometric algebra transformers for large biomedical surface and volume meshes](https://mlsanity.com/paper/2403.07536/) (2024)
- [Diffusion-Based Hierarchical Graph Neural Networks for Simulating Nonlinear Solid Mechanics](https://mlsanity.com/paper/2506.06045/) (2025)
- [MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanics](https://mlsanity.com/paper/2601.23177/) (2026)
- [Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs](https://mlsanity.com/paper/2602.02788/) (2026)
- [DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecasting](https://mlsanity.com/paper/2403.02914/) (2024)
- [Space and Time Continuous Physics Simulation From Partial Observations](https://mlsanity.com/paper/2401.09198/) (2024)
- [PIORF: Physics-Informed Ollivier-Ricci Flow for Long-Range Interactions in Mesh Graph Neural Networks](https://mlsanity.com/paper/2504.04052/) (2025)
- [Improving Long-Range Interactions in Graph Neural Simulators via Hamiltonian Dynamics](https://mlsanity.com/paper/2511.08185/) (2025)
- [Latent Neural PDE Solver: a reduced-order modelling framework for partial differential equations](https://mlsanity.com/paper/2402.17853/) (2024)
- [FLUID-LLM: Learning Computational Fluid Dynamics with Spatiotemporal-aware Large Language Models](https://mlsanity.com/paper/2406.04501/) (2024)
- [PeSANet: Physics-encoded Spectral Attention Network for Simulating PDE-Governed Complex Systems](https://mlsanity.com/paper/2505.01736/) (2025)
- [SlotPi: Physics-informed Object-centric Reasoning Models](https://mlsanity.com/paper/2506.10778/) (2025)
- [Overtone: Cyclic Patch Modulation for Clean, Efficient, and Flexible Physics Emulators](https://mlsanity.com/paper/2507.09264/) (2025)
- [Training Transformers for Mesh-Based Simulations](https://mlsanity.com/paper/2508.18051/) (2025)
- [Towards Universal Mesh Movement Networks](https://mlsanity.com/paper/2407.00382/) (2024)
- [EvoMesh: Adaptive Physical Simulation with Hierarchical Graph Evolutions](https://mlsanity.com/paper/2410.03779/) (2024)
- [Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulence](https://mlsanity.com/paper/2606.01470/) (2026)
- [A Two-Phase Deep Learning Framework for Adaptive Time-Stepping in High-Speed Flow Modeling](https://mlsanity.com/paper/2506.07969/) (2025)
- [M4GN: Mesh-based Multi-segment Hierarchical Graph Network for Dynamic Simulations](https://mlsanity.com/paper/2509.10659/) (2025)

## Abstract (arXiv)

Estimating fluid dynamics is classically done through the simulation and integration of numerical models solving the Navier-Stokes equations, which is computationally complex and time-consuming even on high-end hardware. This is a notoriously hard problem to solve, which has recently been addressed with machine learning, in particular graph neural networks (GNN) and variants trained and evaluated on datasets of static objects in static scenes with fixed geometry. We attempt to go beyond existing work in complexity and introduce a new model, method and benchmark. We propose EAGLE, a large-scale dataset of 1.1 million 2D meshes resulting from simulations of unsteady fluid dynamics caused by a moving flow source interacting with nonlinear scene structure, comprised of 600 different scenes of three different types. To perform future forecasting of pressure and velocity on the challenging EAGLE dataset, we introduce a new mesh transformer. It leverages node clustering, graph pooling and global attention to learn long-range dependencies between spatially distant data points without needing a large number of iterations, as existing GNN methods do. We show that our transformer outperforms state-of-the-art performance on, both, existing synthetic and real datasets and on EAGLE. Finally, we highlight that our approach learns to attend to airflow, integrating complex information in a single iteration.