# FluidFormer: Transformer with Continuous Convolution for Particle-based Fluid Simulation

arXiv: https://arxiv.org/abs/2508.01537 · HTML: https://mlsanity.com/paper/2508.01537/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Rotary Position Embedding for Vision Transformer](https://mlsanity.com/paper/2403.13298/) (2024)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [Deep Dynamical Modeling and Control of Unsteady Fluid Flows](https://mlsanity.com/paper/1805.07472/) (2018)
- [Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics](https://mlsanity.com/paper/2210.06036/) (2022)
- [Transformer with Implicit Edges for Particle-based Physics Simulation](https://mlsanity.com/paper/2207.10860/) (2022)
- [Physics-Incorporated Convolutional Recurrent Neural Networks for Source Identification and Forecasting of Dynamical Systems](https://mlsanity.com/paper/2004.06243/) (2020)
- [DualFluidNet: an Attention-based Dual-pipeline Network for FLuid Simulation](https://mlsanity.com/paper/2312.16867/) (2023)

## Built on by (most cited first, 3 of 3)

- [CORGI: GNNs with Convolutional Residual Global Interactions for Lagrangian Simulation](https://mlsanity.com/paper/2511.22938/) (2025)
- [Attention mechanism for scalable mesh-based neural surrogates of free-surface fluids](https://mlsanity.com/paper/2606.23251/) (2026)
- [Adaptive Interaction Graphs for Particle Simulation](https://mlsanity.com/paper/2609.30822/) (2026)

## Abstract (arXiv)

Learning-based fluid simulation networks have been proven as viable alternatives to traditional numerical solvers for the Navier-Stokes equations. Existing neural methods follow Smoothed Particle Hydrodynamics (SPH) frameworks, which inherently rely only on local inter-particle interactions. However, we emphasize that global context integration is also essential for learning-based methods to stabilize complex fluid simulations. We propose the first Fluid Attention Block (FAB) with a local-global hierarchy, where continuous convolutions extract local features while self-attention captures global dependencies. This fusion suppresses the error accumulation and models long-range physical phenomena. Furthermore, we pioneer the first Transformer architecture specifically designed for continuous fluid simulation, seamlessly integrated within a dual-pipeline architecture. Our method establishes a new paradigm for neural fluid simulation by unifying convolution-based local features with attention-based global context modeling. FluidFormer demonstrates state-of-the-art performance, with stronger stability in complex fluid scenarios.