# On the inductive bias of infinite-depth ResNets and the bottleneck rank

arXiv: https://arxiv.org/abs/2501.19149 · HTML: https://mlsanity.com/paper/2501.19149/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2025.

## Built on by (most cited first, 4 of 4)

- [Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers](https://mlsanity.com/paper/2505.15239/) (2025)
- [Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks](https://mlsanity.com/paper/2606.14954/) (2026)
- [Differentially Private Natural Gradient Descent](https://mlsanity.com/paper/2607.05866/) (2026)
- [Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks](https://mlsanity.com/paper/2607.07884/) (2026)

## Abstract (arXiv)

We compute the minimum-norm weights of a deep linear ResNet, and find that the inductive bias of this architecture lies between minimizing nuclear norm and rank. This implies that, with appropriate hyperparameters, deep nonlinear ResNets have an inductive bias towards minimizing bottleneck rank.