Does Transolver really need a Transformer?
Not on the map. Why?
Abstract (arXiv)
The widely used Transolver family of neural operators is based on physics-attention, which softly assigns the points of an unstructured mesh to a small number of slices, applies self-attention among the resulting tokens, and broadcasts the result back to the points. We provide a comprehensive empirical and theoretical analysis to elucidate the mechanisms which are responsible for model performance. To this end, we perform careful ablations on a challenging suite of nine 3D fluid dynamics benchmarks to find that replacing token attention with a constant linear map does not affect the accuracy. Thus, Transolver does not need a Transformer at all. However, removing the global mixing (slicing/deslicing) or doing it only once leads to performance collapse. We leverage the theory of averaging neural operators to explain and corroborate our findings by showing that just slicing/deslicing, in conjunction with pointwise MLPs, already suffices for universal approximation of continuous operators and attention is redundant in this context. Finally, we provide a novel FlashAttention-style efficient implementation of the key slicing/deslicing module of Transolver. This flashslice kernel streams slice and deslice over the points without materializing the heavy slice-weight tensor, while reproducing the best available implementation to floating point error. At the same time, it leads to very significant memory and compute savings, particularly at large slice counts.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
22 of 31 references.
References · 22
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
- 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
- 2021Choose a Transformer: Fourier or Galerkincited by 343
- 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 336
- 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 326
- 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 303
- 2022Transformer for Partial Differential Equations' Operator Learningcited by 283
- 2023Convolutional Neural Operators for robust and accurate learning of PDEscited by 214
- 2024Poseidon: Efficient Foundation Models for PDEscited by 200
- 2025Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometriescited by 97
10 more
- 2024DrivAerNet++: A Large-Scale Multimodal Car Dataset with Computational Fluid Dynamics Simulations and Deep Learning Benchmarkscited by 75
- 2025Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domainscited by 62
- 2023Nonlocality and Nonlinearity Implies Universality in Operator Learningcited by 43
- 2024DrivAerML: High-Fidelity Computational Fluid Dynamics Dataset for Road-Car External Aerodynamicscited by 35
- 2024Generative AI for fast and accurate statistical computation of fluidscited by 30
- 2025RIGNO: A Graph-based framework for robust and accurate operator learning for PDEs on arbitrary domainscited by 23
- 2025GeoTransolver: Learning Physics on Irregular Domains Using Multi-scale Geometry Aware Physics Attention Transformercited by 23
- 2026Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometriescited by 20
- 2025Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attentioncited by 13
- 2026Simple yet Effective: Low-Rank Spatial Attention for Neural Operatorscited by 4
Built on by
Not cited yet.