# From Taylor Series to Fourier Synthesis: The Periodic Linear Unit

arXiv: https://arxiv.org/abs/2508.01175 · HTML: https://mlsanity.com/paper/2508.01175/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 3 references that are in the map.
- First posted on arXiv 2 Aug 2025.

## Builds on (most cited first, 1 of 1)

- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)

## Abstract (arXiv)

The dominant paradigm in modern neural networks relies on simple, monotonically-increasing activation functions like ReLU. While effective, this paradigm necessitates large, massively-parameterized models to approximate complex functions. In this paper, we introduce the Periodic Linear Unit (PLU), a learnable sine-wave based activation with periodic non-monotonicity. PLU is designed for maximum expressive power and numerical stability, achieved through its formulation and a paired innovation we term Repulsive Reparameterization, which prevents the activation from collapsing into a non-expressive linear function. We demonstrate that a minimal MLP with only two PLU neurons can solve the spiral classification task, a feat impossible for equivalent networks using standard activations. This suggests a paradigm shift from networks as piecewise Taylor-like approximators to powerful Fourier-like function synthesizers, achieving exponential gains in parameter efficiency by placing intelligence in the neuron itself.