# Neuronal architecture extracts statistical temporal patterns

arXiv: https://arxiv.org/abs/2301.10203 · HTML: https://mlsanity.com/paper/2301.10203/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 78 references that are in the map.
- First posted on arXiv 24 Jan 2023.

## Builds on (most cited first, 12 of 12)

- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Benchmarking Multivariate Time Series Classification Algorithms](https://mlsanity.com/paper/2007.13156/) (2020)
- [Activation Functions: Comparison of trends in Practice and Research for Deep Learning](https://mlsanity.com/paper/1811.03378/) (2018)
- [Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks](https://mlsanity.com/paper/2010.09458/) (2020)
- [full-FORCE: A Target-Based Method for Training Recurrent Networks](https://mlsanity.com/paper/1710.03070/) (2017)
- [Transition to chaos in random neuronal networks](https://mlsanity.com/paper/1508.06486/) (2015)
- [Unfolding recurrence by Green's functions for optimized reservoir computing](https://mlsanity.com/paper/2010.06247/) (2020)
- [Emergence of robust memory manifolds](https://mlsanity.com/paper/2109.03879/) (2021)
- [Input correlations impede suppression of chaos and learning in balanced rate networks](https://mlsanity.com/paper/2201.09916/) (2022)
- [Capacity of the covariance perceptron](https://mlsanity.com/paper/1912.00824/) (2019)
- [Goldilocks Neural Networks](https://mlsanity.com/paper/2002.05059/) (2020)

## Abstract (arXiv)

Neuronal systems need to process temporal signals. We here show how higher-order temporal (co-)fluctuations can be employed to represent and process information. Concretely, we demonstrate that a simple biologically inspired feedforward neuronal model is able to extract information from up to the third order cumulant to perform time series classification. This model relies on a weighted linear summation of synaptic inputs followed by a nonlinear gain function. Training both - the synaptic weights and the nonlinear gain function - exposes how the non-linearity allows for the transfer of higher order correlations to the mean, which in turn enables the synergistic use of information encoded in multiple cumulants to maximize the classification accuracy. The approach is demonstrated both on a synthetic and on real world datasets of multivariate time series. Moreover, we show that the biologically inspired architecture makes better use of the number of trainable parameters as compared to a classical machine-learning scheme. Our findings emphasize the benefit of biological neuronal architectures, paired with dedicated learning algorithms, for the processing of information embedded in higher-order statistical cumulants of temporal (co-)fluctuations.