# Fourier Feature Pyramids for Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2605.24278 · HTML: https://mlsanity.com/paper/2605.24278/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 22 May 2026.

## Builds on (most cited first, 28 of 28)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [Towards Understanding the Spectral Bias of Deep Learning](https://mlsanity.com/paper/1912.01198/) (2019)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [BACON: Band-limited Coordinate Networks for Multiscale Scene Representation](https://mlsanity.com/paper/2112.04645/) (2021)
- [Frequency Bias in Neural Networks for Input of Non-Uniform Density](https://mlsanity.com/paper/2003.04560/) (2020)
- [Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks](https://mlsanity.com/paper/2104.08426/) (2021)
- [An Expert's Guide to Training Physics-informed Neural Networks](https://mlsanity.com/paper/2308.08468/) (2023)
- [KAN 2.0: Kolmogorov-Arnold Networks Meet Science](https://mlsanity.com/paper/2408.10205/) (2024)
- [A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks](https://mlsanity.com/paper/2406.02917/) (2024)
- [PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2306.08827/) (2023)
- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [PirateNets: Physics-informed Deep Learning with Residual Adaptive Networks](https://mlsanity.com/paper/2402.00326/) (2024)
- [On the expressiveness and spectral bias of KANs](https://mlsanity.com/paper/2410.01803/) (2024)
- [Asymptotic self-similar blow-up profile for three-dimensional axisymmetric Euler equations using neural networks](https://mlsanity.com/paper/2201.06780/) (2022)
- [FC-PINO: High Precision Physics-Informed Neural Operators via Fourier Continuation](https://mlsanity.com/paper/2211.15960/) (2022)
- [High precision PINNs in unbounded domains: application to singularity formulation in PDEs](https://mlsanity.com/paper/2506.19243/) (2025)
- [Fourier PINNs: From Strong Boundary Conditions to Adaptive Fourier Bases](https://mlsanity.com/paper/2410.03496/) (2024)
- [KANO: Kolmogorov-Arnold Neural Operator](https://mlsanity.com/paper/2509.16825/) (2025)
- [Spectral Informed Neural Network: An Efficient and Low-Memory PINN](https://mlsanity.com/paper/2408.16414/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks](https://mlsanity.com/paper/2609.02110/) (2026)

## Abstract (arXiv)

We present an improved neural field architecture for solving partial differential equations (PDEs). Current physics-informed neural networks (PINNs) provide a flexible framework for solving PDEs, but they struggle to achieve highly accurate solutions and require computation that scales poorly with parameter count. Our model, which we call beignet (Bandlimited Embedding with Interpolated Grid Network), replaces the random Fourier feature embedding used by existing PINN models with a trainable multi-resolution Fourier feature pyramid. To query beignet at a continuous coordinate, we use Fourier interpolation at each level of the pyramid to return features at the input coordinate, and then decode this vector with a fully-connected neural network trunk. Our model provides multiple benefits: 1) Spatial derivatives can be computed efficiently by using the chain rule to compose derivatives of the neural network computed with automatic differentiation with derivatives of the feature grid computed spectrally by the Fast Fourier transform (FFT). 2) beignet can achieve higher accuracy in a compute-efficient manner by scaling the parameter count of this Fourier feature pyramid, instead of the less-efficient strategy of scaling the neural network architecture. 3) beignet can directly control the representation bandlimit, resulting in more stable optimization for difficult PDEs. We demonstrate that beignet finds significantly more accurate solutions on PDE benchmarks using fewer parameters than state-of-the-art PINN methods. We further evaluate beignet on the self-similar inviscid Burgers blowup problem and show that it can minimize residuals to near machine precision using Adam, an accuracy regime previously attained only by using computationally expensive higher-order optimizers.