# Training Deep Fourier Neural Networks To Fit Time-Series Data

arXiv: https://arxiv.org/abs/1405.2262 · HTML: https://mlsanity.com/paper/1405.2262/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2014.

## Built on by (most cited first, 7 of 7)

- [How important are activation functions in regression and classification? A survey, performance comparison, and future directions](https://mlsanity.com/paper/2209.02681/) (2022)
- [Fast Training of Sinusoidal Neural Fields via Scaling Initialization](https://mlsanity.com/paper/2410.04779/) (2024)
- [PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers](https://mlsanity.com/paper/2410.01870/) (2024)
- [Fast-FNet: Accelerating Transformer Encoder Models via Efficient Fourier Layers](https://mlsanity.com/paper/2209.12816/) (2022)
- [Fourier-RNNs for Modelling Noisy Physics Data](https://mlsanity.com/paper/2302.06534/) (2023)
- [Uncertainty Aware Learning from Demonstrations in Multiple Contexts using Bayesian Neural Networks](https://mlsanity.com/paper/1903.05697/) (2019)
- [Neural Fields for Fast and Scalable Interpolation of Geophysical Ocean Variables](https://mlsanity.com/paper/2211.10444/) (2022)

## Abstract (arXiv)

We present a method for training a deep neural network containing sinusoidal activation functions to fit to time-series data. Weights are initialized using a fast Fourier transform, then trained with regularization to improve generalization. A simple dynamic parameter tuning method is employed to adjust both the learning rate and regularization term, such that stability and efficient training are both achieved. We show how deeper layers can be utilized to model the observed sequence using a sparser set of sinusoid units, and how non-uniform regularization can improve generalization by promoting the shifting of weight toward simpler units. The method is demonstrated with time-series problems to show that it leads to effective extrapolation of nonlinear trends.