mlsanityData to 7 Oct 2026

Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries

arXiv 2602.04940 ↗First posted 4 Feb 2026cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Deep learning has emerged as a transformative tool for the neural surrogate modeling of partial differential equations (PDEs), known as neural PDE solvers. However, scaling these solvers to industrial-scale geometries with over 108 cells remains a fundamental challenge due to the prohibitive memory complexity of processing high-resolution meshes. We present Transolver-3, a new member of the Transolver family as a highly scalable framework designed for high-fidelity physics simulations. To bridge the gap between limited GPU capacity and the resolution requirements of complex engineering tasks, we introduce two key architectural optimizations: faster slice and deslice by exploiting matrix multiplication associative property and geometry slice tiling to partition the computation of physical states. Combined with an amortized training strategy by learning on random subsets of original high-resolution meshes and a physical state caching technique during inference, Transolver-3 enables high-fidelity field prediction on industrial-scale meshes. Extensive experiments demonstrate that Transolver-3 can handle meshes with over 160 million cells, achieving impressive performance across three challenging simulation benchmarks, including aircraft and automotive design tasks. Code is available at https://github.com/thuml/Transolver-3.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

30 of 49 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  5. 2020Rethinking Attention with Performerscited by 1,584
  6. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  7. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  8. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  9. 2019Graph U-Netscited by 489
  10. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  11. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  12. 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 321
18 more
  1. 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 299
  2. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  3. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  4. 2024Poseidon: Efficient Foundation Models for PDEscited by 186
  5. 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 137
  6. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 128
  7. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  8. 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 90
  9. 2025Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometriescited by 89
  10. 2023Solving High-Dimensional PDEs with Latent Spectral Modelscited by 84
  11. 2023Improved Operator Learning by Orthogonal Attentioncited by 58
  12. 2025Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domainscited by 56
  13. 2025AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformerscited by 49
  14. 2024DrivAerML: High-Fidelity Computational Fluid Dynamics Dataset for Road-Car External Aerodynamicscited by 33
  15. 2024Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solverscited by 31
  16. 2024RoPINN: Region Optimized Physics-Informed Neural Networkscited by 26
  17. 2024AhmedML: High-Fidelity Computational Fluid Dynamics Dataset for Incompressible, Low-Speed Bluff Body Aerodynamicscited by 13
  18. 2025PhySense: Sensor Placement Optimization for Accurate Physics Sensingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 20.

20 papers.

Sort:

By area:

  1. 2026GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Trainingcited by 8
  2. 2026RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamicscited by 1
  3. 2026ShardTensor: Domain Parallelism for Scientific Machine Learningcited by 1
  4. 2026Adapting Automotive Aerodynamics Surrogates to New Vehicle Families via Transfer Learningcited by 1
  5. 2026Learning Neural Operator Surrogates for the Black Hole Accretion Codecited by 0
  6. 2026M3: Reframing Training Measures for Discretized Physical Simulationscited by 0
  7. 2026Neural Statistical Functionscited by 0
  8. 2026Courant: a State-Adaptive Perceiver-Based Neural Surrogate with Local Support and Interpretable Field Decompositioncited by 0
  9. 2026Let There Be Light: Reflection, Refraction and Scattering for Neural Operatorscited by 0
  10. 2026Attention mechanism for scalable mesh-based neural surrogates of free-surface fluidscited by 0
  11. 2026Physics Transformer: Tailoring Transformer for General PDE Predictioncited by 0
  12. 2026FLARE++: Low-rank attention with dynamic attention routingcited by 0
8 more
  1. 2026HI-MeshGraphNets: Efficient and Accurate Mesh-based Physics Learning with Hierarchical Multi-scale Graph Neural Networkscited by 0
  2. 2026Distance-Aware Attention and Wall-Distance Expert Routing for Transformer-Based 3D Flow Predictioncited by 0
  3. 2026MENO: Memory-Efficient Neural Operatorcited by 0
  4. 2026HGPTrans: Hierarchical Graph-Pooling Transolver for Automotive Aerodynamic Drag Coefficient Predictioncited by 0
  5. 2026Does Transolver really need a Transformer?cited by 0
  6. 2026Neural Harmonic Measure Operatorcited by 0
  7. 2026Transolver-σ: Joint Spectral-Physical Subspace Modeling for Neural PDE Solvingcited by 0
  8. 2026Cross-Domain Pretraining for Steady-State Neural CFD Surrogatescited by 0