I Dropped a Neural Net
Not on the map. Why?
Abstract (arXiv)
A recent Dwarkesh Patel podcast with John Collison and Elon Musk featured an interesting puzzle from Jane Street: they trained a neural net, shuffled all 96 layers, and asked to put them back in order. Given unlabelled layers of a Residual Network and its training dataset, we recover the exact ordering of the layers. The problem decomposes into pairing each block's input and output projections (48! possibilities) and ordering the reassembled blocks (48! possibilities), for a combined search space of (48!)2 ≈ 10122, which is more than the atoms in the observable universe. We show that stability conditions during training like dynamic isometry leave the product Wout Win for correctly paired layers with a negative diagonal structure, allowing us to use diagonal dominance ratio as a signal for pairing. For ordering, we seed-initialize with a rough proxy such as delta-norm or |Wout|F then hill-climb to zero mean squared error.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 4 of its 10 references that are in the map. First posted on arXiv 23 Feb 2026.
Builds on
4 of 10 references.
References · 4
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
- 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
- 2022Rank Diminishing in Deep Neural Networkscited by 47
Built on by
Not cited yet.