mlsanityData to 7 Oct 2026

Transolver: A Fast Transformer Solver for PDEs on General Geometries

arXiv 2402.02366 ↗First posted 4 Feb 2024cs.LG cs.NA math.NAcited by 299

Not on the map. Why?

Abstract (arXiv)

Transformers have empowered many milestones across various fields and have recently been applied to solve partial differential equations (PDEs). However, since PDEs are typically discretized into large-scale meshes with complex geometries, it is challenging for Transformers to capture intricate physical correlations directly from massive individual points. Going beyond superficial and unwieldy meshes, we present Transolver based on a more foundational idea, which is learning intrinsic physical states hidden behind discretized geometries. Specifically, we propose a new Physics-Attention to adaptively split the discretized domain into a series of learnable slices of flexible shapes, where mesh points under similar physical states will be ascribed to the same slice. By calculating attention to physics-aware tokens encoded from slices, Transovler can effectively capture intricate physical correlations under complex geometrics, which also empowers the solver with endogenetic geometry-general modeling capacity and can be efficiently computed in linear complexity. Transolver achieves consistent state-of-the-art with 22% relative gain across six standard benchmarks and also excels in large-scale industrial simulations, including car and airfoil designs. Code is available at https://github.com/thuml/Transolver.

Key facts. Cited by 299 papers in the map (data through 7 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

41 of 63 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  9. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  10. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  11. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  12. 2020Reformer: The Efficient Transformercited by 1,658
29 more
  1. 2020Rethinking Attention with Performerscited by 1,584
  2. 2020Point Transformercited by 1,462
  3. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  4. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  5. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  6. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  7. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  8. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  9. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  10. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  11. 2019Graph U-Netscited by 489
  12. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  13. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  14. 2021Choose a Transformer: Fourier or Galerkincited by 333
  15. 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 321
  16. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  17. 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 292
  18. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  19. 2021Factorized Fourier Neural Operatorscited by 237
  20. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
  21. 2022Patches Are All You Need?cited by 201
  22. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  23. 2020Understanding and mitigating gradient pathologies in physics-informed neural networkscited by 134
  24. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  25. 2023Solving High-Dimensional PDEs with Latent Spectral Modelscited by 84
  26. 2022Deep transfer operator learning for partial differential equations under conditional shiftcited by 83
  27. 2022AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier-Stokes Solutionscited by 77
  28. 2022Flowformer: Linearizing Transformers with Conservation Flowscited by 69
  29. 2023Improved Operator Learning by Orthogonal Attentioncited by 58

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 102; 2026: 170.

299 papers.

Sort:

By area:

  1. 2025AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformerscited by 49
  2. 2024AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fieldscited by 38
  3. 2024DrivAerNet: A Parametric Car Dataset for Data-Driven Aerodynamic Design and Predictioncited by 33
  4. 2024Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solverscited by 31
  5. 2025PDE-Transformer: Efficient and Versatile Transformers for Physics Simulationscited by 29
  6. 2024Wavelet Diffusion Neural Operatorcited by 28
  7. 2025Generative Latent Neural PDE Solver using Flow Matchingcited by 28
  8. 2024Recent Advances on Machine Learning for Computational Fluid Dynamics: A Surveycited by 27
  9. 2025Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraintscited by 26
  10. 2024Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operatorscited by 25
  11. 2024Text2PDE: Latent Diffusion Models for Accessible Physics Simulationcited by 25
  12. 2025Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systemscited by 21
38 more
  1. 2025GeoTransolver: Learning Physics on Irregular Domains Using Multi-scale Geometry Aware Physics Attention Transformercited by 20
  2. 2026Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometriescited by 20
  3. 2024Space-Time Continuous PDE Forecasting using Equivariant Neural Fieldscited by 18
  4. 2024Physics-Informed Geometry-Aware Neural Operatorcited by 18
  5. 2025BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamicscited by 16
  6. 2024Harnessing Scale and Physics: A Multi-Graph Neural Operator Framework for PDEs on Arbitrary Geometriescited by 15
  7. 2025Geometric Operator Learning with Optimal Transportcited by 15
  8. 2025TripNet: Learning Large-scale High-fidelity 3D Car Aerodynamics with Triplane Networkscited by 14
  9. 2024P2C2Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamicscited by 13
  10. 2025FlowDAS: A Stochastic Interpolant-based Framework for Data Assimilationcited by 13
  11. 2026RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Datacited by 13
  12. 2025Active operator learning with predictive uncertainty quantification for partial differential equationscited by 12
  13. 2025PhySense: Sensor Placement Optimization for Accurate Physics Sensingcited by 11
  14. 2025Mesh-Informed Neural Operator : A Transformer Generative Approachcited by 11
  15. 2025Guiding diffusion models to reconstruct flow fields from sparse datacited by 11
  16. 2025Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attentioncited by 11
  17. 2024Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flowscited by 10
  18. 2024Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learningcited by 10
  19. 2024Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Spacecited by 10
  20. 2024NeuralDEM -- Real-time Simulation of Industrial Particulate Flowscited by 10
  21. 2025Towards scalable surrogate models based on Neural Fields for large scale aerodynamic simulationscited by 10
  22. 2025CarBench: A Comprehensive Benchmark for Neural Surrogates on High-Fidelity 3D Car Aerodynamicscited by 10
  23. 2025FlashBias: Fast Computation of Attention with Biascited by 9
  24. 2025PhyDA: Physics-Guided Diffusion Models for Data Assimilation in Atmospheric Systemscited by 9
  25. 2025Turb-L1: Achieving Long-term Turbulence Tracing By Tackling Spectral Biascited by 9
  26. 2024Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogatescited by 8
  27. 2025Latent Mamba Operator for Partial Differential Equationscited by 8
  28. 2025PhysicsCorrect: A Training-Free Approach for Stable Neural PDE Simulationscited by 8
  29. 2026MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanicscited by 8
  30. 2026GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Trainingcited by 8
  31. 2024GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioningcited by 7
  32. 2025TANTE: Time-Adaptive Operator Learning via Neural Taylor Expansioncited by 7
  33. 2025AMR-Transformer: Enabling Efficient Long-range Interaction for Complex Neural Fluid Simulationcited by 7
  34. 2025GyroSwin: 5D Surrogates for Gyrokinetic Plasma Turbulence Simulationscited by 7
  35. 2024M2NO: An Efficient Multi-Resolution Operator Framework for Dynamic Multi-Scale PDE Solverscited by 6
  36. 2025Operator Learning with Domain Decomposition for Geometry Generalization in PDE Solvingcited by 6
  37. 2025Enabling Automatic Differentiation with Mollified Graph Neural Operatorscited by 6
  38. 2025SIMSHIFT: A Benchmark for Adapting Neural Surrogates to Distribution Shiftscited by 6

50 most cited of 299 All 299 (JSON)