mlsanityData to 14 Oct 2026

Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEs

arXiv 2211.15188 ↗First posted 28 Nov 2022cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

Fourier Neural Operators (FNO) offer a principled approach to solving challenging partial differential equations (PDE) such as turbulent flows. At the core of FNO is a spectral layer that leverages a discretization-convergent representation in the Fourier domain, and learns weights over a fixed set of frequencies. However, training FNO presents two significant challenges, particularly in large-scale, high-resolution applications: (i) Computing Fourier transform on high-resolution inputs is computationally intensive but necessary since fine-scale details are needed for solving many PDEs, such as fluid flows, (ii) selecting the relevant set of frequencies in the spectral layers is challenging, and too many modes can lead to overfitting, while too few can lead to underfitting. To address these issues, we introduce the Incremental Fourier Neural Operator (iFNO), which progressively increases both the number of frequency modes used by the model as well as the resolution of the training data. We empirically show that iFNO reduces total training time while maintaining or improving generalization performance across various datasets. Our method demonstrates a 10% lower testing error, using 20% fewer frequency modes compared to the existing Fourier Neural Operator, while also achieving a 30% faster training.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 25 of its 37 references that are in the map. First posted on arXiv 28 Nov 2022.

Builds on

25 of 37 references.

References · 25

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  3. 2018On the Spectral Bias of Neural Networkscited by 1,341
  4. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
  5. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
  6. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  7. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  8. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  9. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
  10. 2021Machine learning accelerated computational fluid dynamicscited by 471
  11. 2021Neural Operator: Learning Maps Between Function Spacescited by 414
  12. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 413
13 more
  1. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  2. 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 297
  3. 2021Curriculum Learning: A Surveycited by 259
  4. 2018Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantificationcited by 199
  5. 2019Towards Understanding the Spectral Bias of Deep Learningcited by 174
  6. 2019Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networkscited by 145
  7. 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 135
  8. 2020The Random Feature Model for Input-Output Maps between Banach Spacescited by 90
  9. 2022Learning Operators with Coupled Attentioncited by 90
  10. 2019Learning to Optimize Multigrid PDE Solverscited by 79
  11. 2019Splitting Steepest Descent for Growing Neural Architecturescited by 55
  12. 2021Spectral Bias in Practice: The Role of Function Frequency in Generalizationcited by 37
  13. 2021PINNup: Robust neural network wavefield solutions using frequency upscaling and neuron splittingcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 11; 2026: 5.

25 papers.

Sort:

By area:

  1. 2024A Library for Learning Neural Operatorscited by 38
  2. 2024Discretization Error of Fourier Neural Operatorscited by 22
  3. 2024PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systemscited by 18
  4. 2025Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learningcited by 12
  5. 2023InRank: Incremental Low-Rank Learningcited by 10
  6. 2024Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Spacecited by 10
  7. 2025The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operatorscited by 8
  8. 2024A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problemscited by 6
  9. 2025TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Trainingcited by 6
  10. 2025Enabling Automatic Differentiation with Mollified Graph Neural Operatorscited by 6
  11. 2023Multi-fidelity Fourier Neural Operator for Fast Modeling of Large-Scale Geological Carbon Storagecited by 5
  12. 2025Bubbleformer: Forecasting Boiling with Transformerscited by 5
13 more
  1. 2025F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learningcited by 3
  2. 2025Method of Manufactured Learning for Solver-free Training of Neural Operatorscited by 3
  3. 2025GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equationscited by 2
  4. 2026Learning Physical Operators using Neural Operatorscited by 2
  5. 2026DGNet: Discrete Green Networks for Data-Efficient Learning of Spatiotemporal PDEscited by 2
  6. 2024Quantum Circuit Training with Growth-Based Architecturescited by 1
  7. 2025Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operatorscited by 1
  8. 2025Accelerating Conjugate Gradient Solvers for Homogenization Problems with Unitary Neural Operatorscited by 1
  9. 2025On the accuracy of implicit neural representations for cardiovascular anatomies and hemodynamic fieldscited by 1
  10. 2026SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operatorscited by 1
  11. 2024Deep Parallel Spectral Neural Operators for Solving Partial Differential Equations with Enhanced Low-Frequency Learning Capabilitycited by 0
  12. 2026Deep Wave Network for Modeling Multi-Scale Physical Dynamicscited by 0
  13. 2026HiLNO: A Hierarchical Latent Neural Operator with Multi-Scale Supervision for PDEs on General Geometriescited by 0