mlsanityData to 7 Oct 2026

Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective

arXiv 2404.07200 ↗First posted 10 Apr 2024cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

In solving partial differential equations (PDEs), Fourier Neural Operators (FNOs) have exhibited notable effectiveness. However, FNO is observed to be ineffective with large Fourier kernels that parameterize more frequencies. Current solutions rely on setting small kernels, restricting FNO's ability to capture complex PDE data in real-world applications. This paper offers empirical insights into FNO's difficulty with large kernels through spectral analysis: FNO exhibits a unique Fourier parameterization bias, excelling at learning dominant frequencies in target data while struggling with non-dominant frequencies. To mitigate such a bias, we propose SpecB-FNO to enhance the capture of non-dominant frequencies by adopting additional residual modules to learn from the previous ones' prediction residuals iteratively. By effectively utilizing large Fourier kernels, SpecB-FNO achieves better prediction accuracy on diverse PDE applications, with an average improvement of 50%.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 32 of its 46 references that are in the map. First posted on arXiv 10 Apr 2024.

Builds on

32 of 46 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  4. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  5. 2018On the Spectral Bias of Neural Networkscited by 1,321
  6. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
  7. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 516
  8. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  9. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  10. 2022Message Passing Neural PDE Solverscited by 368
  11. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 360
  12. 2021Choose a Transformer: Fourier or Galerkincited by 333
20 more
  1. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  2. 2021Factorized Fourier Neural Operatorscited by 237
  3. 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 225
  4. 2022U-NO: U-shaped Neural Operatorscited by 219
  5. 2023Convolutional Neural Operators for robust and accurate learning of PDEscited by 207
  6. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 190
  7. 2022Clifford Neural Layers for PDE Modelingcited by 91
  8. 2022Learning Operators with Coupled Attentioncited by 88
  9. 2023Operator Learning with Neural Fields: Tackling PDEs on General Geometriescited by 80
  10. 2023Neural Inverse Operators for Solving PDE Inverse Problemscited by 61
  11. 2021Seismic wave propagation and inversion with Neural Operatorscited by 51
  12. 2022Mitigating spectral bias for the multiscale operator learningcited by 43
  13. 2023Group Equivariant Fourier Neural Operators for Partial Differential Equationscited by 42
  14. 2023MgNO: Efficient Parameterization of Linear Operators via Multigridcited by 40
  15. 2023Domain Agnostic Fourier Neural Operatorscited by 36
  16. 2024BENO: Boundary-embedded Neural Operators for Elliptic PDEscited by 22
  17. 2022Guiding continuous operator learning through Physics-based boundary constraintscited by 21
  18. 2022Transform Once: Efficient Operator Learning in Frequency Domaincited by 20
  19. 2023Deep Equilibrium Based Neural Operators for Steady-State PDEscited by 19
  20. 2023Guaranteed Approximation Bounds for Mixed-Precision Neural Operatorscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 15.

24 papers.

Sort:

By area:

  1. 2025Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarkscited by 10
  2. 2024Optimal Convergence Rates for Neural Operatorscited by 7
  3. 2024Modeling Multivariable High-resolution 3D Urban Microclimate Using Localized Fourier Neural Operatorcited by 6
  4. 2025KANO: Kolmogorov-Arnold Neural Operatorcited by 6
  5. 2024Spectral Convolutional Conditional Neural Processescited by 4
  6. 2026MENO: MeanFlow-Enhanced Neural Operators for Dynamical Systemscited by 3
  7. 2025Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Featurescited by 2
  8. 2026PDEInvBench: A Comprehensive Dataset and Design Space Exploration of Neural Networks for PDE Inverse Problemscited by 2
  9. 2026A Physics-Informed Fourier-Wavelet Transformer for Multiscale Computational Fluid Dynamics Surrogate Modelingcited by 2
  10. 2025Towards Robust Surrogate Models: Benchmarking Machine Learning Approaches to Expediting Phase Field Simulations of Brittle Fracturecited by 1
  11. 2026Unveiling the Mechanism of Continuous Representation Full-Waveform Inversion: A Wave Based Neural Tangent Kernel Frameworkcited by 1
  12. 2026Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigationcited by 1
12 more
  1. 2026SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operatorscited by 1
  2. 2026Operator Boosting Produces Pareto-Efficient PDE Surrogatescited by 1
  3. 2025Multiscale Neural PDE Surrogates for Prediction and Downscaling: Application to Ocean Currentscited by 0
  4. 2025Real-time nonlinear inversion of magnetic resonance elastography with operator learningcited by 0
  5. 2026Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operatorscited by 0
  6. 2026Mitigating Frequency Learning Bias in Quantum Models via Multi-Stage Residual Learningcited by 0
  7. 2026Derived Fields Preserve Fine-Scale Detail in Budgeted Neural Simulatorscited by 0
  8. 2026Correcting Neural Operator Spectral Bias via Diffusion Posterior Sampling with Sparse Observationscited by 0
  9. 2026PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogatescited by 0
  10. 2026Read, Write, Relax: Why Neural PDE Surrogates Need Both Global and Local Processingcited by 0
  11. 2026Learning Metamaterial Eigenmodes with Wavelet-Encoded Fourier Neural Operatorscited by 0
  12. 2026Periodic Neural Mapping for Unsteady Rotor-Blade Pressure and Aeroelastic Load Predictioncited by 0