# AMR-Transformer: Enabling Efficient Long-range Interaction for Complex Neural Fluid Simulation

arXiv: https://arxiv.org/abs/2503.10257 · HTML: https://mlsanity.com/paper/2503.10257/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Scalable Transformer for PDE Surrogate Modeling](https://mlsanity.com/paper/2305.17560/) (2023)
- [Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs](https://mlsanity.com/paper/2310.00120/) (2023)
- [Sequential Deep Operator Networks (S-DeepONet) for Predicting Full-field Solutions Under Time-dependent Loads](https://mlsanity.com/paper/2306.08218/) (2023)
- [CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamics](https://mlsanity.com/paper/2310.05963/) (2023)
- [Simulating Continuum Mechanics with Multi-Scale Graph Neural Networks](https://mlsanity.com/paper/2106.04900/) (2021)
- [Transfer Learning Enhanced DeepONet for Long-Time Prediction of Evolution Equations](https://mlsanity.com/paper/2212.04663/) (2022)
- [Constraint-based graph network simulator](https://mlsanity.com/paper/2112.09161/) (2021)
- [Large-scale graph representation learning with very deep GNNs and self-supervision](https://mlsanity.com/paper/2107.09422/) (2021)
- [Multi-scale Time-stepping of Partial Differential Equations with Transformers](https://mlsanity.com/paper/2311.02225/) (2023)
- [A multi-fidelity deep operator network (DeepONet) for fusing simulation and monitoring data: Application to real-time settlement prediction during tunnel construction](https://mlsanity.com/paper/2310.00057/) (2023)
- [Fluid Simulation on Neural Flow Maps](https://mlsanity.com/paper/2312.14635/) (2023)
- [Multiscale graph neural networks with adaptive mesh refinement for accelerating mesh-based simulations](https://mlsanity.com/paper/2402.08863/) (2024)
- [Fourier PINNs: From Strong Boundary Conditions to Adaptive Fourier Bases](https://mlsanity.com/paper/2410.03496/) (2024)
- [Learning Physical Simulation with Message Passing Transformer](https://mlsanity.com/paper/2406.06060/) (2024)

## Built on by (most cited first, 8 of 8)

- [When do World Models Successfully Learn Dynamical Systems?](https://mlsanity.com/paper/2507.04898/) (2025)
- [Shock-Aware Physics-Guided Fusion-DeepONet Operator for Rarefied Micro-Nozzle Flows](https://mlsanity.com/paper/2510.17887/) (2025)
- [LLM4Fluid: Large Language Models as Generalizable Neural Solvers for Fluid Dynamics](https://mlsanity.com/paper/2601.21681/) (2026)
- [In Transformer We Trust? A Perspective on Transformer Architecture Failure Modes](https://mlsanity.com/paper/2602.14318/) (2026)
- [Compressing History into Memory: Distilling Transformers into Recurrent Transformers](https://mlsanity.com/paper/2606.21562/) (2026)
- [Learning Neural Operator Surrogates for the Black Hole Accretion Code](https://mlsanity.com/paper/2604.25985/) (2026)
- [MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE Transformers](https://mlsanity.com/paper/2606.04366/) (2026)
- [Dynamic Quadtree Tokenization and Transformer for Adaptive Mesh PDE Forecasting](https://mlsanity.com/paper/2610.04044/) (2026)

## Abstract (arXiv)

Accurately and efficiently simulating complex fluid dynamics is a challenging task that has traditionally relied on computationally intensive methods. Neural network-based approaches, such as convolutional and graph neural networks, have partially alleviated this burden by enabling efficient local feature extraction. However, they struggle to capture long-range dependencies due to limited receptive fields, and Transformer-based models, while providing global context, incur prohibitive computational costs. To tackle these challenges, we propose AMR-Transformer, an efficient and accurate neural CFD-solving pipeline that integrates a novel adaptive mesh refinement scheme with a Navier-Stokes constraint-aware fast pruning module. This design encourages long-range interactions between simulation cells and facilitates the modeling of global fluid wave patterns, such as turbulence and shockwaves. Experiments show that our approach achieves significant gains in efficiency while preserving critical details, making it suitable for high-resolution physical simulations with long-range dependencies. On CFDBench, PDEBench and a new shockwave dataset, our pipeline demonstrates up to an order-of-magnitude improvement in accuracy over baseline models. Additionally, compared to ViT, our approach achieves a reduction in FLOPs of up to 60 times.