MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attention
Not on the map. Why?
Abstract (arXiv)
A key scalability challenge in neural solvers for industrial-scale physics simulations is efficiently capturing both fine-grained local interactions and long-range global dependencies across millions of spatial elements. We introduce the Multi-Scale Patch Transformer (MSPT), an architecture that combines local point attention within patches with global attention to coarse patch-level representations. To partition the input domain into spatially-coherent patches, we employ ball trees, which handle irregular geometries efficiently. This dual-scale design enables MSPT to scale to millions of points on a single GPU. We validate our method on standard PDE benchmarks (elasticity, plasticity, fluid dynamics, porous flow) and large-scale aerodynamic datasets (ShapeNet-Car, Ahmed-ML), achieving state-of-the-art accuracy with substantially lower memory footprint and computational cost.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
32 of 48 references.
References · 32
- 2017Attention Is All You Needcited by 59,932
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
- 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
- 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
- 2019Graph U-Netscited by 489
- 2023Symbolic Discovery of Optimization Algorithmscited by 423
- 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
20 more
- 2021Choose a Transformer: Fourier or Galerkincited by 333
- 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 321
- 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 299
- 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
- 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 292
- 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
- 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 237
- 2021Factorized Fourier Neural Operatorscited by 237
- 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
- 2022U-NO: U-shaped Neural Operatorscited by 219
- 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
- 2025Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometriescited by 89
- 2023Solving High-Dimensional PDEs with Latent Spectral Modelscited by 84
- 2024Latent Neural Operator for Solving Forward and Inverse PDE Problemscited by 76
- 2023Improved Operator Learning by Orthogonal Attentioncited by 58
- 2025AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformerscited by 49
- 2024CViT: Continuous Vision Transformer for Operator Learningcited by 47
- 2021PatchFormer: An Efficient Point Transformer with Patch Attentioncited by 43
- 2025Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systemscited by 21
- 2024AhmedML: High-Fidelity Computational Fluid Dynamics Dataset for Incompressible, Low-Speed Bluff Body Aerodynamicscited by 13
Built on by
Citing papers per year: 2026: 3.
3 papers.