On the inductive bias of infinite-depth ResNets and the bottleneck rank
Not on the map. Why?
Abstract (arXiv)
We compute the minimum-norm weights of a deep linear ResNet, and find that the inductive bias of this architecture lies between minimizing nuclear norm and rank. This implies that, with appropriate hyperparameters, deep nonlinear ResNets have an inductive bias towards minimizing bottleneck rank.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Jan 2025.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2025: 1; 2026: 3.
4 papers.
- 2025Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformerscited by 10
- 2026Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networkscited by 2
- 2026Differentially Private Natural Gradient Descentcited by 0
- 2026Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networkscited by 0
- 2026Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks8 Jul 2026
- 2026Differentially Private Natural Gradient Descent7 Jul 2026
- 2026Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks12 Jun 2026
- 2025Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers21 May 2025