# Complex Recurrent Spectral Network

arXiv: https://arxiv.org/abs/2312.07296 · HTML: https://mlsanity.com/paper/2312.07296/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 63 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 23 of 23)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network](https://mlsanity.com/paper/1707.02485/) (2017)
- [Cognitive computational neuroscience](https://mlsanity.com/paper/1807.11819/) (2018)
- [Unveiling the structure of wide flat minima in neural networks](https://mlsanity.com/paper/2107.01163/) (2021)
- [Learning through atypical "phase transitions" in overparameterized neural networks](https://mlsanity.com/paper/2110.00683/) (2021)
- [Machine learning in spectral domain](https://mlsanity.com/paper/2005.14436/) (2020)
- [The backtracking survey propagation algorithm for solving random K-SAT problems](https://mlsanity.com/paper/1508.05117/) (2015)
- [Deep learning via message passing algorithms based on belief propagation](https://mlsanity.com/paper/2110.14583/) (2021)
- [Learning from Survey Propagation: a Neural Network for MAX-E-3-SAT](https://mlsanity.com/paper/2012.06344/) (2020)
- [Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes](https://mlsanity.com/paper/2012.07747/) (2020)
- [Phase transitions in the mini-batch size for sparse and dense two-layer neural networks](https://mlsanity.com/paper/2305.06435/) (2023)
- [Hard Optimization Problems have Soft Edges](https://mlsanity.com/paper/2209.04824/) (2022)
- [How a student becomes a teacher: learning and forgetting through Spectral methods](https://mlsanity.com/paper/2310.12612/) (2023)
- [Large independent sets on random d-regular graphs with fixed degree d](https://mlsanity.com/paper/2003.12293/) (2020)
- [Recurrent Spectral Network (RSN): shaping the basin of attraction of a discrete map to reach automated classification](https://mlsanity.com/paper/2202.04497/) (2022)
- [Chaos and correlated avalanches in excitatory neural networks with synaptic plasticity](https://mlsanity.com/paper/1610.00252/) (2016)
- [On the training of sparse and dense deep neural networks: less parameters, same performance](https://mlsanity.com/paper/2106.09021/) (2021)
- [Parallel Learning by Multitasking Neural Networks](https://mlsanity.com/paper/2308.04106/) (2023)
- [Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems](https://mlsanity.com/paper/2309.05337/) (2023)

## Built on by (most cited first, 5 of 5)

- [Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach](https://mlsanity.com/paper/2406.01183/) (2024)
- [Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise](https://mlsanity.com/paper/2409.13470/) (2024)
- [Learning in Wilson-Cowan model for metapopulation](https://mlsanity.com/paper/2406.16453/) (2024)
- [Engineered Ordinary Differential Equations as Classification Algorithm (EODECA): thorough characterization and testing](https://mlsanity.com/paper/2312.14681/) (2023)
- [A short review on the maximum clique problem algorithms with classical, AI, and quantum methods](https://mlsanity.com/paper/2403.09742/) (2024)

## Abstract (arXiv)

This paper presents a novel approach to advancing artificial intelligence (AI) through the development of the Complex Recurrent Spectral Network (mathbbC-RSN), an innovative variant of the Recurrent Spectral Network (RSN) model. The mathbbC-RSN is designed to address a critical limitation in existing neural network models: their inability to emulate the complex processes of biological neural networks dynamically and accurately. By integrating key concepts from dynamical systems theory and leveraging principles from statistical mechanics, the mathbbC-RSN model introduces localized non-linearity, complex fixed eigenvalues, and a distinct separation of memory and input processing functionalities. These features collectively enable the mathbbC-RSN evolving towards a dynamic, oscillating final state that more closely mirrors biological cognition. Central to this work is the exploration of how the mathbbC-RSN manages to capture the rhythmic, oscillatory dynamics intrinsic to biological systems, thanks to its complex eigenvalue structure and the innovative segregation of its linear and non-linear components. The model's ability to classify data through a time-dependent function, and the localization of information processing, is demonstrated with an empirical evaluation using the MNIST dataset. Remarkably, distinct items supplied as a sequential input yield patterns in time which bear the indirect imprint of the insertion order (and of the time of separation between contiguous insertions).