# Self-organized criticality in neural networks

arXiv: https://arxiv.org/abs/2107.03402 · HTML: https://mlsanity.com/paper/2107.03402/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 21 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Builds on (most cited first, 3 of 3)

- [The world as a neural network](https://mlsanity.com/paper/2008.01540/) (2020)
- [Towards a theory of machine learning](https://mlsanity.com/paper/2004.09280/) (2020)
- [Emergent Quantumness in Neural Networks](https://mlsanity.com/paper/2012.05082/) (2020)

## Built on by (most cited first, 1 of 1)

- [Bio-inspired Machine Learning: programmed death and replication](https://mlsanity.com/paper/2207.04886/) (2022)

## Abstract (arXiv)

We demonstrate, both analytically and numerically, that learning dynamics of neural networks is generically attracted towards a self-organized critical state. The effect can be modeled with quartic interactions between non-trainable variables (e.g. states of neurons) and trainable variables (e.g. weight matrix). Non-trainable variables are rapidly driven towards stochastic equilibrium and trainable variables are slowly driven towards learning equilibrium described by a scale-invariant distribution on a wide range of scales. Our results suggest that the scale invariance observed in many physical and biological systems might be due to some kind of learning dynamics and support the claim that the universe might be a neural network.