# Infinite-dimensional Folded-in-time Deep Neural Networks

arXiv: https://arxiv.org/abs/2101.02966 · HTML: https://mlsanity.com/paper/2101.02966/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 75 references that are in the map.
- First posted on arXiv 8 Jan 2021.

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [A Survey of Neuromorphic Computing and Neural Networks in Hardware](https://mlsanity.com/paper/1705.06963/) (2017)
- [Learning One-Shot Imitation from Humans without Humans](https://mlsanity.com/paper/1911.01103/) (2019)
- [Deep Time-Delay Reservoir Computing: Dynamics and Memory Capacity](https://mlsanity.com/paper/2006.06322/) (2020)
- [Performance boost of time-delay reservoir computing by non-resonant clock cycle](https://mlsanity.com/paper/1905.02534/) (2019)
- [Deep Function Machines: Generalized Neural Networks for Topological Layer Expression](https://mlsanity.com/paper/1612.04799/) (2016)
- [Reservoir computing with simple oscillators: Virtual and real networks](https://mlsanity.com/paper/1802.08590/) (2018)
- [Limitations of the recall capabilities in delay based reservoir computing systems](https://mlsanity.com/paper/2010.15562/) (2020)

## Abstract (arXiv)

The method recently introduced in arXiv:2011.10115 realizes a deep neural network with just a single nonlinear element and delayed feedback. It is applicable for the description of physically implemented neural networks. In this work, we present an infinite-dimensional generalization, which allows for a more rigorous mathematical analysis and a higher flexibility in choosing the weight functions. Precisely speaking, the weights are described by Lebesgue integrable functions instead of step functions. We also provide a functional back-propagation algorithm, which enables gradient descent training of the weights. In addition, with a slight modification, our concept realizes recurrent neural networks.