Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries
Not on the map. Why?
Abstract (arXiv)
Deep learning has emerged as a transformative tool for the neural surrogate modeling of partial differential equations (PDEs), known as neural PDE solvers. However, scaling these solvers to industrial-scale geometries with over 108 cells remains a fundamental challenge due to the prohibitive memory complexity of processing high-resolution meshes. We present Transolver-3, a new member of the Transolver family as a highly scalable framework designed for high-fidelity physics simulations. To bridge the gap between limited GPU capacity and the resolution requirements of complex engineering tasks, we introduce two key architectural optimizations: faster slice and deslice by exploiting matrix multiplication associative property and geometry slice tiling to partition the computation of physical states. Combined with an amortized training strategy by learning on random subsets of original high-resolution meshes and a physical state caching technique during inference, Transolver-3 enables high-fidelity field prediction on industrial-scale meshes. Extensive experiments demonstrate that Transolver-3 can handle meshes with over 160 million cells, achieving impressive performance across three challenging simulation benchmarks, including aircraft and automotive design tasks. Code is available at https://github.com/thuml/Transolver-3.
Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
30 of 49 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2020Rethinking Attention with Performerscited by 1,584
- 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
- 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
- 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
- 2019Graph U-Netscited by 489
- 2023Symbolic Discovery of Optimization Algorithmscited by 423
- 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
- 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 321
18 more
- 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 299
- 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
- 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
- 2024Poseidon: Efficient Foundation Models for PDEscited by 186
- 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 137
- 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 128
- 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
- 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 90
- 2025Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometriescited by 89
- 2023Solving High-Dimensional PDEs with Latent Spectral Modelscited by 84
- 2023Improved Operator Learning by Orthogonal Attentioncited by 58
- 2025Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domainscited by 56
- 2025AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformerscited by 49
- 2024DrivAerML: High-Fidelity Computational Fluid Dynamics Dataset for Road-Car External Aerodynamicscited by 33
- 2024Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solverscited by 31
- 2024RoPINN: Region Optimized Physics-Informed Neural Networkscited by 26
- 2024AhmedML: High-Fidelity Computational Fluid Dynamics Dataset for Incompressible, Low-Speed Bluff Body Aerodynamicscited by 13
- 2025PhySense: Sensor Placement Optimization for Accurate Physics Sensingcited by 11
Built on by
Citing papers per year: 2026: 20.
20 papers.
Sort:
By area:
- 2026GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Trainingcited by 8
- 2026RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamicscited by 1
- 2026ShardTensor: Domain Parallelism for Scientific Machine Learningcited by 1
- 2026Adapting Automotive Aerodynamics Surrogates to New Vehicle Families via Transfer Learningcited by 1
- 2026Learning Neural Operator Surrogates for the Black Hole Accretion Codecited by 0
- 2026M3: Reframing Training Measures for Discretized Physical Simulationscited by 0
- 2026Neural Statistical Functionscited by 0
- 2026Courant: a State-Adaptive Perceiver-Based Neural Surrogate with Local Support and Interpretable Field Decompositioncited by 0
- 2026Let There Be Light: Reflection, Refraction and Scattering for Neural Operatorscited by 0
- 2026Attention mechanism for scalable mesh-based neural surrogates of free-surface fluidscited by 0
- 2026Physics Transformer: Tailoring Transformer for General PDE Predictioncited by 0
- 2026FLARE++: Low-rank attention with dynamic attention routingcited by 0
8 more
- 2026HI-MeshGraphNets: Efficient and Accurate Mesh-based Physics Learning with Hierarchical Multi-scale Graph Neural Networkscited by 0
- 2026Distance-Aware Attention and Wall-Distance Expert Routing for Transformer-Based 3D Flow Predictioncited by 0
- 2026MENO: Memory-Efficient Neural Operatorcited by 0
- 2026HGPTrans: Hierarchical Graph-Pooling Transolver for Automotive Aerodynamic Drag Coefficient Predictioncited by 0
- 2026Does Transolver really need a Transformer?cited by 0
- 2026Neural Harmonic Measure Operatorcited by 0
- 2026Transolver-σ: Joint Spectral-Physical Subspace Modeling for Neural PDE Solvingcited by 0
- 2026Cross-Domain Pretraining for Steady-State Neural CFD Surrogatescited by 0
- 2026Cross-Domain Pretraining for Steady-State Neural CFD Surrogates7 Oct 2026
- 2026Transolver-σ: Joint Spectral-Physical Subspace Modeling for Neural PDE Solving29 Sep 2026
- 2026Neural Harmonic Measure Operator28 Sep 2026
- 2026Does Transolver really need a Transformer?26 Sep 2026
- 2026HGPTrans: Hierarchical Graph-Pooling Transolver for Automotive Aerodynamic Drag Coefficient Prediction24 Sep 2026
- 2026MENO: Memory-Efficient Neural Operator23 Sep 2026
- 2026Distance-Aware Attention and Wall-Distance Expert Routing for Transformer-Based 3D Flow Prediction7 Sep 2026
- 2026HI-MeshGraphNets: Efficient and Accurate Mesh-based Physics Learning with Hierarchical Multi-scale Graph Neural Networks13 Aug 2026
- 2026FLARE++: Low-rank attention with dynamic attention routing12 Aug 2026
- 2026Physics Transformer: Tailoring Transformer for General PDE Prediction27 Jul 2026
- 2026Attention mechanism for scalable mesh-based neural surrogates of free-surface fluids22 Jun 2026
- 2026Let There Be Light: Reflection, Refraction and Scattering for Neural Operators2 Jun 2026
8 more
- 2026Adapting Automotive Aerodynamics Surrogates to New Vehicle Families via Transfer Learning27 May 2026
- 2026Courant: a State-Adaptive Perceiver-Based Neural Surrogate with Local Support and Interpretable Field Decomposition24 May 2026
- 2026Neural Statistical Functions11 May 2026
- 2026ShardTensor: Domain Parallelism for Scientific Machine Learning11 May 2026
- 2026M3: Reframing Training Measures for Discretized Physical Simulations9 May 2026
- 2026RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamics30 Apr 2026
- 2026Learning Neural Operator Surrogates for the Black Hole Accretion Code28 Apr 2026
- 2026GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Training23 Feb 2026
- 2026GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Trainingcited by 8
- 2026RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamicscited by 1
- 2026ShardTensor: Domain Parallelism for Scientific Machine Learningcited by 1
- 2026Adapting Automotive Aerodynamics Surrogates to New Vehicle Families via Transfer Learningcited by 1
- 2026Learning Neural Operator Surrogates for the Black Hole Accretion Codecited by 0
- 2026M3: Reframing Training Measures for Discretized Physical Simulationscited by 0
- 2026Neural Statistical Functionscited by 0
- 2026Courant: a State-Adaptive Perceiver-Based Neural Surrogate with Local Support and Interpretable Field Decompositioncited by 0
- 2026Let There Be Light: Reflection, Refraction and Scattering for Neural Operatorscited by 0
- 2026Attention mechanism for scalable mesh-based neural surrogates of free-surface fluidscited by 0
- 2026Physics Transformer: Tailoring Transformer for General PDE Predictioncited by 0
- 2026FLARE++: Low-rank attention with dynamic attention routingcited by 0
12 most cited of 19 All 19 (JSON)
- 2026Cross-Domain Pretraining for Steady-State Neural CFD Surrogates7 Oct 2026
- 2026Transolver-σ: Joint Spectral-Physical Subspace Modeling for Neural PDE Solving29 Sep 2026
- 2026Neural Harmonic Measure Operator28 Sep 2026
- 2026Does Transolver really need a Transformer?26 Sep 2026
- 2026HGPTrans: Hierarchical Graph-Pooling Transolver for Automotive Aerodynamic Drag Coefficient Prediction24 Sep 2026
- 2026MENO: Memory-Efficient Neural Operator23 Sep 2026
- 2026HI-MeshGraphNets: Efficient and Accurate Mesh-based Physics Learning with Hierarchical Multi-scale Graph Neural Networks13 Aug 2026
- 2026FLARE++: Low-rank attention with dynamic attention routing12 Aug 2026
- 2026Physics Transformer: Tailoring Transformer for General PDE Prediction27 Jul 2026
- 2026Attention mechanism for scalable mesh-based neural surrogates of free-surface fluids22 Jun 2026
- 2026Let There Be Light: Reflection, Refraction and Scattering for Neural Operators2 Jun 2026
- 2026Adapting Automotive Aerodynamics Surrogates to New Vehicle Families via Transfer Learning27 May 2026
12 newest of 19 All 19 (JSON)