# Neural Spectral Methods: Self-supervised learning in the spectral domain

arXiv: https://arxiv.org/abs/2312.05225 · HTML: https://mlsanity.com/paper/2312.05225/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations](https://mlsanity.com/paper/2203.01360/) (2022)
- [From Fourier to Koopman: Spectral Methods for Long-term Time Series Prediction](https://mlsanity.com/paper/2004.00574/) (2020)
- [Learning to correct spectral methods for simulating turbulent flows](https://mlsanity.com/paper/2207.00556/) (2022)
- [Applications of physics informed neural operators](https://mlsanity.com/paper/2203.12634/) (2022)
- [Accelerated Training of Physics-Informed Neural Networks (PINNs) using Meshless Discretizations](https://mlsanity.com/paper/2205.09332/) (2022)
- [Hybrid FEM-NN models: Combining artificial neural networks with the finite element method](https://mlsanity.com/paper/2101.00962/) (2021)
- [Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Control](https://mlsanity.com/paper/2105.14094/) (2021)
- [Transform Once: Efficient Operator Learning in Frequency Domain](https://mlsanity.com/paper/2211.14453/) (2022)
- [Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problems](https://mlsanity.com/paper/2202.02710/) (2022)

## Built on by (most cited first, 14 of 14)

- [Scaling physics-informed hard constraints with mixture-of-experts](https://mlsanity.com/paper/2402.13412/) (2024)
- [Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows](https://mlsanity.com/paper/2405.17211/) (2024)
- [Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarks](https://mlsanity.com/paper/2504.07099/) (2025)
- [Calibrated Physics-Informed Uncertainty Quantification](https://mlsanity.com/paper/2502.04406/) (2025)
- [EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale](https://mlsanity.com/paper/2510.24173/) (2025)
- [Continuum Transformers Perform In-Context Learning by Operator Gradient Descent](https://mlsanity.com/paper/2505.17838/) (2025)
- [Neuro-Spectral Architectures for Causal Physics-Informed Networks](https://mlsanity.com/paper/2509.04966/) (2025)
- [M2M: Learning controllable Multi of experts and multi-scale operators are the Partial Differential Equations need](https://mlsanity.com/paper/2410.11617/) (2024)
- [PDEInvBench: A Comprehensive Dataset and Design Space Exploration of Neural Networks for PDE Inverse Problems](https://mlsanity.com/paper/2605.25353/) (2026)
- [Physics-Informed Heterogeneous Graph Neural Networks for DC Blocker Placement](https://mlsanity.com/paper/2405.10389/) (2024)
- [Spectral-inspired Operator Learning with Limited Data and Unknown Physics](https://mlsanity.com/paper/2505.21573/) (2025)
- [Distribution-Free Robust Predict-Then-Optimize in Function Spaces](https://mlsanity.com/paper/2602.08215/) (2026)
- [Mitigating Gradient Pathology in PINNs through Aligned Constraint](https://mlsanity.com/paper/2605.25001/) (2026)
- [Perturbative-NeuSA: A Structured Spectral Framework for Time-Dependent PDEs](https://mlsanity.com/paper/2607.24345/) (2026)

## Abstract (arXiv)

We present Neural Spectral Methods, a technique to solve parametric Partial Differential Equations (PDEs), grounded in classical spectral methods. Our method uses orthogonal bases to learn PDE solutions as mappings between spectral coefficients. In contrast to current machine learning approaches which enforce PDE constraints by minimizing the numerical quadrature of the residuals in the spatiotemporal domain, we leverage Parseval's identity and introduce a new training strategy through a textitspectral loss. Our spectral loss enables more efficient differentiation through the neural network, and substantially reduces training complexity. At inference time, the computational cost of our method remains constant, regardless of the spatiotemporal resolution of the domain. Our experimental results demonstrate that our method significantly outperforms previous machine learning approaches in terms of speed and accuracy by one to two orders of magnitude on multiple different problems. When compared to numerical solvers of the same accuracy, our method demonstrates a 10× increase in performance speed.