# A Novel Fourier Feature Network for Solving Partial Differential Equations

arXiv: https://arxiv.org/abs/2608.14733 · HTML: https://mlsanity.com/paper/2608.14733/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 13 Aug 2026.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Normalization Techniques in Training DNNs: Methodology, Analysis and Application](https://mlsanity.com/paper/2009.12836/) (2020)
- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2012.02895/) (2020)
- [On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements](https://mlsanity.com/paper/2110.14121/) (2021)
- [Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine](https://mlsanity.com/paper/2204.11375/) (2022)
- [Fourier PINNs: From Strong Boundary Conditions to Adaptive Fourier Bases](https://mlsanity.com/paper/2410.03496/) (2024)
- [Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth](https://mlsanity.com/paper/2211.14503/) (2022)

## Abstract (arXiv)

Building on the foundation of single-hidden-layer neural networks, Fourier Feature Networks (FENs) are proposed, which incorporate Fourier features using cos, sin, or a combination of both. Similar to Extreme Learning Machines (ELMs), FENs employ a single-hidden-layer architecture to generate a set of basis functions. The target function is then approximated as a linear combination of these basis functions, with the coefficients determined using the least squares method. However, unlike ELMs, which often rely on affine transformations to improve representational power, FENs can achieve high-precision solutions without requiring such transformations on the input variables. To evaluate the representational capacity of these networks, we search for an optimal scaling factor within a predefined range for the randomly initialized and fixed weights and biases. By adjusting this scaling factor, we ensure a fair comparison between FENs and ELMs using various activation functions, such as sigmoid, tanh, and swish. Our numerical experiments demonstrate that FENs consistently achieve higher accuracy than ELMs.