mlsanityData to 14 Oct 2026

Randomly initialized autoencoders: fixed points and edge-of-chaos

arXiv 2608.14638 ↗First posted 28 Jul 2026cs.LG math.PR math.STcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises questions of existence, stability, and basins of attraction of these fixed points. These questions are addressed via the contractive properties of autoencoders, and are closely related to the notion of edge-of-chaos. Edge-of-chaos (EoC) is an important notion in the theory of DNNs. It describes the critical regime separating ordered and chaotic signal propagation through a randomly initialized network. Initialization at or near this critical regime offers several theoretical and practical advantages, including stability of the network w.r.t. perturbations of the input. EoC was previously introduced for broad classes of neural networks using mean-field averaging methods. In this paper we modify the notion of EoC for the study of autoencoders. Specifically, we introduce local and global EoC for autoencoders that control local (small) and global (arbitrary) perturbations of the input respectively. The study of stability of autoencoders falls within the scope of nonlinear problems in Random Matrix Theory (RMT). Our analysis of local EoC is based on spectral techniques of RMT, whereas global EoC is studied by employing Sudakov-Fernique inequality for Gaussian processes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

17 of 32 references.

References · 17

  1. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  2. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  3. 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
  4. 2019Robust Learning with Jacobian Regularizationcited by 129
  5. 2018Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?cited by 94
  6. 2018The Emergence of Spectral Universality in Deep Networkscited by 74
  7. 2018Products of Many Large Random Matrices and Gradients in Deep Neural Networkscited by 49
  8. 2020On Random Matrices Arising in Deep Neural Networks. Gaussian Casecited by 14
  9. 2020On Random Matrices Arising in Deep Neural Networks: General I.I.D. Casecited by 10
  10. 2022Improving Deep Neural Network Random Initialization Through Neuronal Rewiringcited by 7
  11. 2023On the Initialisation of Wide Low-Rank Feedforward Neural Networkscited by 5
  12. 2023Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processescited by 3
5 more
  1. 2023Learning fixed points of recurrent neural networks by reparameterizing the network modelcited by 2
  2. 2026How Controlling the Variance can Improve Training Stability of Sparsely Activated DNNs and CNNscited by 2
  3. 2024Linear and non-linear integrate-and-fire neurons driven by synaptic shot noise with reversal potentialscited by 1
  4. 2025Random weights of DNNs and emergence of fixed pointscited by 1
  5. 2026Macroscopic dynamics of quadratic integrate-and-fire neurons subject to correlated noisecited by 1

Built on by

Not cited yet.