# Deep neural networks from the perspective of ergodic theory

arXiv: https://arxiv.org/abs/2308.03888 · HTML: https://mlsanity.com/paper/2308.03888/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 13 references that are in the map.
- First posted on arXiv 4 Aug 2023.

## Built on by (most cited first, 2 of 2)

- [Description of the Training Process of Neural Networks via Ergodic Theorem : Ghost nodes](https://mlsanity.com/paper/2507.01003/) (2025)
- [Exploding and vanishing gradients in deep neural networks: the effect of residual connections](https://mlsanity.com/paper/2606.17013/) (2026)

## Abstract (arXiv)

The design of deep neural networks remains somewhat of an art rather than precise science. By tentatively adopting ergodic theory considerations on top of viewing the network as the time evolution of a dynamical system, with each layer corresponding to a temporal instance, we show that some rules of thumb, which might otherwise appear mysterious, can be attributed heuristics.