# MeshMask: Physics-Based Simulations with Masked Graph Neural Networks

arXiv: https://arxiv.org/abs/2501.08738 · HTML: https://mlsanity.com/paper/2501.08738/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 15 Jan 2025.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [MultiScale MeshGraphNets](https://mlsanity.com/paper/2210.00612/) (2022)
- [tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow](https://mlsanity.com/paper/1801.09710/) (2018)
- [Simulating Continuum Mechanics with Multi-Scale Graph Neural Networks](https://mlsanity.com/paper/2106.04900/) (2021)
- [MGAE: Masked Autoencoders for Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2201.02534/) (2022)
- [U-net architectures for fast prediction of incompressible laminar flows](https://mlsanity.com/paper/1910.13532/) (2019)
- [Multi-Grid Graph Neural Networks with Self-Attention for Computational Mechanics](https://mlsanity.com/paper/2409.11899/) (2024)

## Built on by (most cited first, 9 of 9)

- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [AneumoBench: A Source-Linked Benchmark for Synthetic-Geometry Transfer in Aneurysm CFD](https://mlsanity.com/paper/2505.14717/) (2025)
- [Graph Deep Learning for Intracranial Aneurysm Blood Flow Simulation and Risk Assessment](https://mlsanity.com/paper/2512.09013/) (2025)
- [A Physics-Informed Fourier-Wavelet Transformer for Multiscale Computational Fluid Dynamics Surrogate Modeling](https://mlsanity.com/paper/2606.24696/) (2026)
- [Curriculum Learning for Mesh-based simulations](https://mlsanity.com/paper/2509.13138/) (2025)
- [Learning Transient Convective Heat Transfer with Geometry Aware World Models](https://mlsanity.com/paper/2601.22086/) (2026)
- [Automated discovery of finite volume schemes using Graph Neural Networks](https://mlsanity.com/paper/2508.19052/) (2025)
- [OceanDepths: A Global Dataset of Paired Subsurface and Surface Ocean Observations](https://mlsanity.com/paper/2608.16373/) (2026)
- [ONE CYLinder: A Benchmark for Graph-Based Surrogate Modeling of Unsteady Bluff-Body Flows](https://mlsanity.com/paper/2609.08947/) (2026)

## Abstract (arXiv)

We introduce a novel masked pre-training technique for graph neural networks (GNNs) applied to computational fluid dynamics (CFD) problems. By randomly masking up to 40% of input mesh nodes during pre-training, we force the model to learn robust representations of complex fluid dynamics. We pair this masking strategy with an asymmetric encoder-decoder architecture and gated multi-layer perceptrons to further enhance performance. The proposed method achieves state-of-the-art results on seven CFD datasets, including a new challenging dataset of 3D intracranial aneurysm simulations with over 250,000 nodes per mesh. Moreover, it significantly improves model performance and training efficiency across such diverse range of fluid simulation tasks. We demonstrate improvements of up to 60% in long-term prediction accuracy compared to previous best models, while maintaining similar computational costs. Notably, our approach enables effective pre-training on multiple datasets simultaneously, significantly reducing the time and data required to achieve high performance on new tasks. Through extensive ablation studies, we provide insights into the optimal masking ratio, architectural choices, and training strategies.