# Image classification in frequency domain with 2SReLU: a second harmonics superposition activation function

arXiv: https://arxiv.org/abs/2006.10853 · HTML: https://mlsanity.com/paper/2006.10853/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 8 of 8)

- [As large as it gets: Learning infinitely large Filters via Neural Implicit Functions in the Fourier Domain](https://mlsanity.com/paper/2307.10001/) (2023)
- [Frequency learning for structured CNN filters with Gaussian fractional derivatives](https://mlsanity.com/paper/2111.06660/) (2021)
- [DC is all you need: describing ReLU from a signal processing standpoint](https://mlsanity.com/paper/2407.16556/) (2024)
- [Learning Convolutional Neural Networks in the Frequency Domain](https://mlsanity.com/paper/2204.06718/) (2022)
- [FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive Review](https://mlsanity.com/paper/2505.13461/) (2025)
- [Frequency Domain Convolutional Neural Network: Accelerated CNN for Large Diabetic Retinopathy Image Classification](https://mlsanity.com/paper/2106.12736/) (2021)
- [Joint Learning of Frequency and Spatial Domains for Dense Predictions](https://mlsanity.com/paper/2202.08991/) (2022)
- [TFDMNet: A Novel Network Structure Combines the Time Domain and Frequency Domain Features](https://mlsanity.com/paper/2401.15949/) (2024)

## Abstract (arXiv)

Deep Convolutional Neural Networks are able to identify complex patterns and perform tasks with super-human capabilities. However, besides the exceptional results, they are not completely understood and it is still impractical to hand-engineer similar solutions. In this work, an image classification Convolutional Neural Network and its building blocks are described from a frequency domain perspective. Some network layers have established counterparts in the frequency domain like the convolutional and pooling layers. We propose the 2SReLU layer, a novel non-linear activation function that preserves high frequency components in deep networks. It is demonstrated that in the frequency domain it is possible to achieve competitive results without using the computationally costly convolution operation. A source code implementation in PyTorch is provided at: https://gitlab.com/thomio/2srelu