# I Dropped a Neural Net

arXiv: https://arxiv.org/abs/2602.19845 · HTML: https://mlsanity.com/paper/2602.19845/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 10 references that are in the map.
- First posted on arXiv 23 Feb 2026.

## Builds on (most cited first, 4 of 4)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks](https://mlsanity.com/paper/1806.05393/) (2018)
- [Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice](https://mlsanity.com/paper/1711.04735/) (2017)
- [Rank Diminishing in Deep Neural Networks](https://mlsanity.com/paper/2206.06072/) (2022)

## Abstract (arXiv)

A recent Dwarkesh Patel podcast with John Collison and Elon Musk featured an interesting puzzle from Jane Street: they trained a neural net, shuffled all 96 layers, and asked to put them back in order. Given unlabelled layers of a Residual Network and its training dataset, we recover the exact ordering of the layers. The problem decomposes into pairing each block's input and output projections (48! possibilities) and ordering the reassembled blocks (48! possibilities), for a combined search space of (48!)2 ≈ 10122, which is more than the atoms in the observable universe. We show that stability conditions during training like dynamic isometry leave the product Wout Win for correctly paired layers with a negative diagonal structure, allowing us to use diagonal dominance ratio as a signal for pairing. For ordering, we seed-initialize with a rough proxy such as delta-norm or |Wout|F then hill-climb to zero mean squared error.