# Exploiting Chaotic Dynamics as Deep Neural Networks

arXiv: https://arxiv.org/abs/2406.02580 · HTML: https://mlsanity.com/paper/2406.02580/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 73 references that are in the map.
- First posted on arXiv 29 May 2024.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Exponential expressivity in deep neural networks through transient chaos](https://mlsanity.com/paper/1606.05340/) (2016)
- [Lipschitz Recurrent Neural Networks](https://mlsanity.com/paper/2006.12070/) (2020)
- [Physical reservoir computing -- An introductory perspective](https://mlsanity.com/paper/2005.00992/) (2020)
- [Lyapunov spectra of chaotic recurrent neural networks](https://mlsanity.com/paper/2006.02427/) (2020)
- [A recurrent neural network without chaos](https://mlsanity.com/paper/1612.06212/) (2016)
- [ChaosNet: A Chaos based Artificial Neural Network Architecture for Classification](https://mlsanity.com/paper/1910.02423/) (2019)
- [Biologically inspired ChaosNet architecture for Hypothetical Protein Classification](https://mlsanity.com/paper/2302.02427/) (2023)
- [Transient chaotic dimensionality expansion by recurrent networks](https://mlsanity.com/paper/2002.11006/) (2020)

## Built on by (most cited first, 2 of 2)

- [Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians](https://mlsanity.com/paper/2505.19458/) (2025)
- [Designing Chaotic Attractors: A Semi-supervised Approach](https://mlsanity.com/paper/2407.09545/) (2024)

## Abstract (arXiv)

Chaos presents complex dynamics arising from nonlinearity and a sensitivity to initial states. These characteristics suggest a depth of expressivity that underscores their potential for advanced computational applications. However, strategies to effectively exploit chaotic dynamics for information processing have largely remained elusive. In this study, we reveal that the essence of chaos can be found in various state-of-the-art deep neural networks. Drawing inspiration from this revelation, we propose a novel method that directly leverages chaotic dynamics for deep learning architectures. Our approach is systematically evaluated across distinct chaotic systems. In all instances, our framework presents superior results to conventional deep neural networks in terms of accuracy, convergence speed, and efficiency. Furthermore, we found an active role of transient chaos formation in our scheme. Collectively, this study offers a new path for the integration of chaos, which has long been overlooked in information processing, and provides insights into the prospective fusion of chaotic dynamics within the domains of machine learning and neuromorphic computation.