Exploding and vanishing gradients in deep neural networks: the effect of residual connections
Not on the map. Why?
Abstract (arXiv)
The well known phenomenon of exploding and vanishing gradients in deep neural networks is analyzed using multiplicative ergodic theory. The effect of adding a residual connection is explained in this context. Specifically, a characterization of Liapunov exponents due to Furstenberg and Kifer is exploited in order to make a precise statement about the Liapunov spectrum and the effect of residual connections on it.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 5 of its 17 references that are in the map. First posted on arXiv 15 Jun 2026.
Builds on
5 of 17 references.
References · 5
- 2018Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?cited by 94
- 2019What Can ResNet Learn Efficiently, Going Beyond Kernels?cited by 91
- 2018Products of Many Large Random Matrices and Gradients in Deep Neural Networkscited by 49
- 2025Residual connections provably mitigate oversmoothing in graph neural networkscited by 7
- 2023Deep neural networks from the perspective of ergodic theorycited by 2
Built on by
Not cited yet.