mlsanityData to 7 Oct 2026

GNOT: A General Neural Operator Transformer for Operator Learning

arXiv 2302.14376 ↗First posted 28 Feb 2023cs.LG cs.NA math.NAcited by 321

Not on the map. Why?

Abstract (arXiv)

Learning partial differential equations' (PDEs) solution operators is an essential problem in machine learning. However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution. To address these challenges, we propose a general neural operator transformer (GNOT), a scalable and effective transformer-based framework for learning operators. By designing a novel heterogeneous normalized attention layer, our model is highly flexible to handle multiple input functions and irregular meshes. Besides, we introduce a geometric gating mechanism which could be viewed as a soft domain decomposition to solve the multi-scale problems. The large model capacity of the transformer architecture grants our model the possibility to scale to large datasets and practical problems. We conduct extensive experiments on multiple challenging datasets from different domains and achieve a remarkable improvement compared with alternative methods. Our code and data are publicly available at urlhttps://github.com/thu-ml/GNOT.

Key facts. Cited by 321 papers in the map (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

26 of 37 references.

References · 26

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  7. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  8. 2020Reformer: The Efficient Transformercited by 1,658
  9. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  10. 2020Rethinking Attention with Performerscited by 1,584
  11. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  12. 2020Efficient Transformers: A Surveycited by 784
14 more
  1. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  2. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  3. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  4. 2021Random Feature Attentioncited by 350
  5. 2021Choose a Transformer: Fourier or Galerkincited by 333
  6. 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 292
  7. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  8. 2021Factorized Fourier Neural Operatorscited by 237
  9. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
  10. 2022MIONet: Learning multiple-input operators via tensor productcited by 180
  11. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  12. 2021Improved architectures and training algorithms for deep operator networkscited by 85
  13. 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 43
  14. 2022Variable-Input Deep Operator Networkscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 80; 2025: 101; 2026: 120.

321 papers.

Sort:

By area:

  1. 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 299
  2. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  3. 2023Neural Operators for Accelerating Scientific Simulations and Designcited by 218
  4. 2024Poseidon: Efficient Foundation Models for PDEscited by 186
  5. 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 137
  6. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  7. 2023Operator Learning with Neural Fields: Tackling PDEs on General Geometriescited by 80
  8. 2023A Mathematical Guide to Operator Learningcited by 74
  9. 2023Improved Operator Learning by Orthogonal Attentioncited by 58
  10. 2025AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformerscited by 49
  11. 2024CViT: Continuous Vision Transformer for Operator Learningcited by 47
  12. 2024Continuum Attention for Neural Operatorscited by 39
38 more
  1. 2024AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fieldscited by 38
  2. 2023Domain Agnostic Fourier Neural Operatorscited by 36
  3. 2024Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modelingcited by 36
  4. 2024On the Benefits of Memory for Modeling Time-Dependent PDEscited by 30
  5. 2023ViTO: Vision Transformer-Operatorcited by 29
  6. 2024Wavelet Diffusion Neural Operatorcited by 28
  7. 2025Generative Latent Neural PDE Solver using Flow Matchingcited by 28
  8. 2024UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptationcited by 27
  9. 2024Recent Advances on Machine Learning for Computational Fluid Dynamics: A Surveycited by 27
  10. 2024State-space models are accurate and efficient neural operators for dynamical systemscited by 27
  11. 2024Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operatorscited by 25
  12. 2023Physics Informed Token Transformer for Solving Partial Differential Equationscited by 21
  13. 2025Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systemscited by 21
  14. 2025GeoTransolver: Learning Physics on Irregular Domains Using Multi-scale Geometry Aware Physics Attention Transformercited by 20
  15. 2026Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometriescited by 20
  16. 2023Deep Equilibrium Based Neural Operators for Steady-State PDEscited by 19
  17. 2024Derivative-enhanced Deep Operator Networkcited by 19
  18. 2025Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulationcited by 19
  19. 2024Peridynamic Neural Operators: A Data-Driven Nonlocal Constitutive Model for Complex Material Responsescited by 18
  20. 2024Invertible Fourier Neural Operators for Tackling Both Forward and Inverse Problemscited by 18
  21. 2024Physics-Informed Geometry-Aware Neural Operatorcited by 18
  22. 2024Nonlocal Attention Operator: Materializing Hidden Knowledge Towards Interpretable Physics Discoverycited by 18
  23. 2024Operator Learning with Gaussian Processescited by 18
  24. 2024Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learningcited by 17
  25. 2024Basis-to-Basis Operator Learning Using Function Encoderscited by 17
  26. 2023Real-time Inference and Extrapolation via a Diffusion-inspired Temporal Transformer Operator (DiTTO)cited by 16
  27. 2023Inducing Point Operator Transformer: A Flexible and Scalable Architecture for Solving PDEscited by 16
  28. 2024Mamba Neural Operator: Who Wins? Transformers vs. State-Space Models for PDEscited by 16
  29. 2023NUNO: A General Framework for Learning Parametric PDEs with Non-Uniform Datacited by 15
  30. 2024Diffeomorphism Neural Operator for various domains and parameters of partial differential equationscited by 15
  31. 2024Harnessing Scale and Physics: A Multi-Graph Neural Operator Framework for PDEs on Arbitrary Geometriescited by 15
  32. 2025Geometric Operator Learning with Optimal Transportcited by 15
  33. 2023Multi-scale Time-stepping of Partial Differential Equations with Transformerscited by 14
  34. 2023Machine learning and domain decomposition methods -- a surveycited by 14
  35. 2024DIMON: Learning Solution Operators of Partial Differential Equations on a Diffeomorphic Family of Domainscited by 14
  36. 2025Operator Learning: A Statistical Perspectivecited by 14
  37. 2025AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific Machine Learningcited by 13
  38. 2026RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Datacited by 13

50 most cited of 321 All 321 (JSON)