# Deep Oscillatory Neural Network

arXiv: https://arxiv.org/abs/2405.03725 · HTML: https://mlsanity.com/paper/2405.03725/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 106 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 14 of 14)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks](https://mlsanity.com/paper/2403.14302/) (2024)
- [Music Generation by Deep Learning - Challenges and Directions](https://mlsanity.com/paper/1712.04371/) (2017)
- [High-performance deep spiking neural networks with 0.3 spikes per neuron](https://mlsanity.com/paper/2306.08744/) (2023)
- [Neural Machine Translation: A Review of Methods, Resources, and Tools](https://mlsanity.com/paper/2012.15515/) (2020)
- [Learning to match transient sound events using attentional similarity for few-shot sound recognition](https://mlsanity.com/paper/1812.01269/) (2018)
- [A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs](https://mlsanity.com/paper/1901.00945/) (2019)
- [Enhancing Adaptive History Reserving by Spiking Convolutional Block Attention Module in Recurrent Neural Networks](https://mlsanity.com/paper/2401.03719/) (2024)
- [Sentiment Analysis Using Simplified Long Short-term Memory Recurrent Neural Networks](https://mlsanity.com/paper/2005.03993/) (2020)

## Built on by (most cited first, 3 of 3)

- [Phasor Agents: Oscillatory Graphs with Three-Factor Plasticity and Sleep-Staged Learning](https://mlsanity.com/paper/2601.04362/) (2026)
- [Graph Coloring Approach to Solving Sudoku with Oscillatory Neural Networks](https://mlsanity.com/paper/2607.15814/) (2026)
- [Frequency-Based Reservoir computing](https://mlsanity.com/paper/2607.24420/) (2026)

## Abstract (arXiv)

We propose a novel, brain-inspired deep neural network model known as the Deep Oscillatory Neural Network (DONN). Deep neural networks like the Recurrent Neural Networks indeed possess sequence processing capabilities but the internal states of the network are not designed to exhibit brain-like oscillatory activity. With this motivation, the DONN is designed to have oscillatory internal dynamics. Neurons of the DONN are either nonlinear neural oscillators or traditional neurons with sigmoidal or ReLU activation. The neural oscillator used in the model is the Hopf oscillator, with the dynamics described in the complex domain. Input can be presented to the neural oscillator in three possible modes. The sigmoid and ReLU neurons also use complex-valued extensions. All the weight stages are also complex-valued. Training follows the general principle of weight change by minimizing the output error and therefore has an overall resemblance to complex backpropagation. A generalization of DONN to convolutional networks known as the Oscillatory Convolutional Neural Network is also proposed. The two proposed oscillatory networks are applied to a variety of benchmark problems in signal and image/video processing. The performance of the proposed models is either comparable or superior to published results on the same data sets.