# Multi-Grid Graph Neural Networks with Self-Attention for Computational Mechanics

arXiv: https://arxiv.org/abs/2409.11899 · HTML: https://mlsanity.com/paper/2409.11899/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 18 Sep 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Self-Attention Graph Pooling](https://mlsanity.com/paper/1904.08082/) (2019)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [Understanding Attention and Generalization in Graph Neural Networks](https://mlsanity.com/paper/1905.02850/) (2019)
- [Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows](https://mlsanity.com/paper/1810.08217/) (2018)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [MultiScale MeshGraphNets](https://mlsanity.com/paper/2210.00612/) (2022)
- [Attending to Graph Transformers](https://mlsanity.com/paper/2302.04181/) (2023)
- [Learning Algebraic Multigrid Using Graph Neural Networks](https://mlsanity.com/paper/2003.05744/) (2020)
- [tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow](https://mlsanity.com/paper/1801.09710/) (2018)
- [Simulating Continuum Mechanics with Multi-Scale Graph Neural Networks](https://mlsanity.com/paper/2106.04900/) (2021)
- [MGAE: Masked Autoencoders for Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2201.02534/) (2022)
- [U-net architectures for fast prediction of incompressible laminar flows](https://mlsanity.com/paper/1910.13532/) (2019)
- [MG-GNN: Multigrid Graph Neural Networks for Learning Multilevel Domain Decomposition Methods](https://mlsanity.com/paper/2301.11378/) (2023)

## Built on by (most cited first, 9 of 9)

- [MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanics](https://mlsanity.com/paper/2601.23177/) (2026)
- [MeshMask: Physics-Based Simulations with Masked Graph Neural Networks](https://mlsanity.com/paper/2501.08738/) (2025)
- [EvoMesh: Adaptive Physical Simulation with Hierarchical Graph Evolutions](https://mlsanity.com/paper/2410.03779/) (2024)
- [Curriculum Learning for Mesh-based simulations](https://mlsanity.com/paper/2509.13138/) (2025)
- [Mask-Morph Graph U-Net: A Generalisable Mesh-Based Surrogate for Crashworthiness Field Prediction under Large Geometric Variation](https://mlsanity.com/paper/2605.15231/) (2026)
- [Automated discovery of finite volume schemes using Graph Neural Networks](https://mlsanity.com/paper/2508.19052/) (2025)
- [Multi-fidelity graph-based neural networks architectures to learn Navier-Stokes solutions on non-parametrized 2D domains](https://mlsanity.com/paper/2601.02157/) (2026)
- [Mesh Based Simulations with Spatial and Temporal awareness](https://mlsanity.com/paper/2605.01542/) (2026)
- [Physics-Informed Coarsening for Multigrid Graph Neural Surrogates](https://mlsanity.com/paper/2605.31013/) (2026)

## Abstract (arXiv)

Advancement in finite element methods have become essential in various disciplines, and in particular for Computational Fluid Dynamics (CFD), driving research efforts for improved precision and efficiency. While Convolutional Neural Networks (CNNs) have found success in CFD by mapping meshes into images, recent attention has turned to leveraging Graph Neural Networks (GNNs) for direct mesh processing. This paper introduces a novel model merging Self-Attention with Message Passing in GNNs, achieving a 15% reduction in RMSE on the well known flow past a cylinder benchmark. Furthermore, a dynamic mesh pruning technique based on Self-Attention is proposed, that leads to a robust GNN-based multigrid approach, also reducing RMSE by 15%. Additionally, a new self-supervised training method based on BERT is presented, resulting in a 25% RMSE reduction. The paper includes an ablation study and outperforms state-of-the-art models on several challenging datasets, promising advancements similar to those recently achieved in natural language and image processing. Finally, the paper introduces a dataset with meshes larger than existing ones by at least an order of magnitude. Code and Datasets will be released at https://github.com/DonsetPG/multigrid-gnn.