# Randomly initialized autoencoders: fixed points and edge-of-chaos

arXiv: https://arxiv.org/abs/2608.14638 · HTML: https://mlsanity.com/paper/2608.14638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Exponential expressivity in deep neural networks through transient chaos](https://mlsanity.com/paper/1606.05340/) (2016)
- [Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice](https://mlsanity.com/paper/1711.04735/) (2017)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?](https://mlsanity.com/paper/1801.03744/) (2018)
- [The Emergence of Spectral Universality in Deep Networks](https://mlsanity.com/paper/1802.09979/) (2018)
- [Products of Many Large Random Matrices and Gradients in Deep Neural Networks](https://mlsanity.com/paper/1812.05994/) (2018)
- [On Random Matrices Arising in Deep Neural Networks. Gaussian Case](https://mlsanity.com/paper/2001.06188/) (2020)
- [On Random Matrices Arising in Deep Neural Networks: General I.I.D. Case](https://mlsanity.com/paper/2011.11439/) (2020)
- [Improving Deep Neural Network Random Initialization Through Neuronal Rewiring](https://mlsanity.com/paper/2207.08148/) (2022)
- [On the Initialisation of Wide Low-Rank Feedforward Neural Networks](https://mlsanity.com/paper/2301.13710/) (2023)
- [Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes](https://mlsanity.com/paper/2310.16597/) (2023)
- [Learning fixed points of recurrent neural networks by reparameterizing the network model](https://mlsanity.com/paper/2307.06732/) (2023)
- [How Controlling the Variance can Improve Training Stability of Sparsely Activated DNNs and CNNs](https://mlsanity.com/paper/2602.05779/) (2026)
- [Linear and non-linear integrate-and-fire neurons driven by synaptic shot noise with reversal potentials](https://mlsanity.com/paper/2403.07670/) (2024)
- [Random weights of DNNs and emergence of fixed points](https://mlsanity.com/paper/2501.04182/) (2025)
- [Macroscopic dynamics of quadratic integrate-and-fire neurons subject to correlated noise](https://mlsanity.com/paper/2601.10032/) (2026)

## Abstract (arXiv)

In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises questions of existence, stability, and basins of attraction of these fixed points. These questions are addressed via the contractive properties of autoencoders, and are closely related to the notion of edge-of-chaos. Edge-of-chaos (EoC) is an important notion in the theory of DNNs. It describes the critical regime separating ordered and chaotic signal propagation through a randomly initialized network. Initialization at or near this critical regime offers several theoretical and practical advantages, including stability of the network w.r.t. perturbations of the input. EoC was previously introduced for broad classes of neural networks using mean-field averaging methods. In this paper we modify the notion of EoC for the study of autoencoders. Specifically, we introduce local and global EoC for autoencoders that control local (small) and global (arbitrary) perturbations of the input respectively. The study of stability of autoencoders falls within the scope of nonlinear problems in Random Matrix Theory (RMT). Our analysis of local EoC is based on spectral techniques of RMT, whereas global EoC is studied by employing Sudakov-Fernique inequality for Gaussian processes.