mlsanityData to 14 Oct 2026

Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs

arXiv 2606.28122 ↗First posted 26 Jun 2026cs.CE cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural operators provide deep neural networks for learning mappings between function spaces. Among them, the Fourier Neural Operator (FNO) is particularly effective: its spectral convolution relies on low-dimensional Fourier-domain representations and can handle inputs at different resolutions. This design aligns well with settings where the Fourier basis diagonalizes the underlying operator, such as linear, constant-coefficient PDEs on periodic domains, in which Fourier modes evolve independently. However, nonlinear PDEs may benefit from an additional inductive bias, as they exhibit structured interactions between modes, governed by polynomial nonlinearities. To capture this inductive bias, we introduce the Higher-Order Spectral Convolution, a spectral mixer that extends FNO from diagonal modulation to explicit n-linear mode mixing, aligned with the dynamics of nonlinear PDEs. Our experiments on standard benchmarks show that the proposed Higher-Order FNO (HO-FNO) retains the efficiency of FNO-based architectures and consistently improves over other spectral neural operators. HO-FNO also performs on par with or better than state-of-the-art transformers and state-space models on several datasets, with stronger gains in highly nonlinear regimes, such as the Poisson equation with polynomial forcing, where a single HO-FNO layer outperforms FNO models with up to 16 layers. We open-source our code for reproducibility at: https://github.com/AlexColagrande/HO-FNO.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 60 references that are in the map. First posted on arXiv 26 Jun 2026.

Builds on

36 of 60 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  6. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  7. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 427
  8. 2021Choose a Transformer: Fourier or Galerkincited by 343
  9. 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 336
  10. 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 326
  11. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 303
  12. 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 297
24 more
  1. 2021Factorized Fourier Neural Operatorscited by 249
  2. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 234
  3. 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 233
  4. 2022U-NO: U-shaped Neural Operatorscited by 230
  5. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solverscited by 216
  6. 2024The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learningcited by 145
  7. 2023Representational Strengths and Limitations of Transformerscited by 133
  8. 2023Scalable Transformer for PDE Surrogate Modelingcited by 128
  9. 2025Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometriescited by 97
  10. 2023Solving High-Dimensional PDEs with Latent Spectral Modelscited by 89
  11. 2023Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEscited by 68
  12. 2023Improved Operator Learning by Orthogonal Attentioncited by 61
  13. 2023Towards Stability of Autoregressive Neural Operatorscited by 59
  14. 2021Systematic Generalization with Edge Transformerscited by 38
  15. 2024A Library for Learning Neural Operatorscited by 38
  16. 2025Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systemscited by 22
  17. 2025Geometric Operator Learning with Optimal Transportcited by 16
  18. 2025Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learningcited by 12
  19. 2025Latent Mamba Operator for Partial Differential Equationscited by 10
  20. 2019Logic and the 2-Simplicial Transformercited by 9
  21. 2025Fast and Simplex: 2-Simplicial Attention in Tritoncited by 8
  22. 2025Linear Attention with Global Context: A Multipole Attention Mechanism for Vision and Physicscited by 5
  23. 2024Convolution goes higher-order: a biologically inspired mechanism empowers image classificationcited by 4
  24. 2025MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attentioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FreqMoE: Dynamic Frequency Enhancement for Neural PDE Solvers
  2. 2025Mitigating Spectral Bias in Neural Operators via High-Frequency Scaling for Physical Systems
  3. 2026Correcting Neural Operator Spectral Bias via Diffusion Posterior Sampling with Sparse Observations
  4. 2026Deep Spectral Learning of Embedded Latent Transfer Operators for Stochastic Dynamical Systems
  5. 2024From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems
  6. 2026Windowed Fourier Propagator: A Frequency-Local Neural Operator for Wave Equations in Inhomogeneous Media
  7. 2024Fourier Neural Operators for Learning Dynamics in Quantum Spin Systems
  8. 2024Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling