# Deep Neural Networks using a Single Neuron: Folded-in-Time Architecture using Feedback-Modulated Delay Loops

arXiv: https://arxiv.org/abs/2011.10115 · HTML: https://mlsanity.com/paper/2011.10115/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [To See a World in a Spark of Neuron: Disentangling Multi-task Interference for Training-free Model Merging](https://mlsanity.com/paper/2503.05320/) (2025)
- [A Generic Shared Attention Mechanism for Various Backbone Neural Networks](https://mlsanity.com/paper/2210.16101/) (2022)
- [Physical Reservoir Computing Enabled by Solitary Waves and Biologically-Inspired Nonlinear Transformation of Input Data](https://mlsanity.com/paper/2402.03319/) (2024)
- [Master memory function for delay-based reservoir computers with single-variable dynamics](https://mlsanity.com/paper/2108.12643/) (2021)
- [Learning Time Delay Systems with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2206.14288/) (2022)
- [Deepening Neural Networks Implicitly and Locally via Recurrent Attention Strategy](https://mlsanity.com/paper/2210.15676/) (2022)
- [Prospective Messaging: Learning in Networks with Communication Delays](https://mlsanity.com/paper/2407.05494/) (2024)
- [SSSUMO: Real-Time Semi-Supervised Submovement Decomposition](https://mlsanity.com/paper/2507.08028/) (2025)
- [Reconstructing Spiking Neural Networks Using a Single Neuron with Autapses](https://mlsanity.com/paper/2603.24692/) (2026)

## Abstract (arXiv)

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.