# Deep Learning without Global Optimization by Random Fourier Neural Networks

arXiv: https://arxiv.org/abs/2407.11894 · HTML: https://mlsanity.com/paper/2407.11894/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Towards Understanding the Spectral Bias of Deep Learning](https://mlsanity.com/paper/1912.01198/) (2019)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Frequency Bias in Neural Networks for Input of Non-Uniform Density](https://mlsanity.com/paper/2003.04560/) (2020)
- [Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding](https://mlsanity.com/paper/2106.02795/) (2021)
- [A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics](https://mlsanity.com/paper/1904.04326/) (2019)
- [Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality](https://mlsanity.com/paper/2106.08900/) (2021)
- [On the Activation Function Dependence of the Spectral Bias of Neural Networks](https://mlsanity.com/paper/2208.04924/) (2022)
- [Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Control](https://mlsanity.com/paper/2105.14094/) (2021)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [Approximation Power of Deep Neural Networks: an explanatory mathematical survey](https://mlsanity.com/paper/2207.09511/) (2022)
- [Comparing Spectral Bias and Robustness For Two-Layer Neural Networks: SGD vs Adaptive Random Fourier Features](https://mlsanity.com/paper/2402.00332/) (2024)
- [Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces](https://mlsanity.com/paper/2405.06727/) (2024)

## Built on by (most cited first, 1 of 1)

- [Noise-adapted Neural Operator for Robust Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2508.09655/) (2025)

## Abstract (arXiv)

We introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov Chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions.