mlsanityData to 7 Oct 2026

Growing a Neural Network in Breadth, Depth, and Time

arXiv 2605.25174 ↗First posted 24 May 2026q-bio.NC cs.LG cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Spatial and temporal resource constraints are critical for both biological and artificial intelligent systems. Here we define differentiable cost terms for breadth, depth, and time within a recurrent convolutional neural network conceived as a finite subset of an infinite lattice. We optimize these costs jointly with task errors via backpropagation. We set different pressures on breadth, depth, and time, which leads to diverse computational graphs emerging organically through training. We find that all three resources can be traded off against each other to achieve a given level of accuracy. Networks grow in all three dimensions with task complexity and spontaneously take more recurrent steps when inputs are occluded. Surprisingly, time used by the model correlates with human reaction times in an object recognition task. Our framework provides a normative account of how resource constraints shape neural architectures, connecting to questions about brain design in neuroscience, and may help illuminate the diversity of neural solutions found in nature.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 41 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

17 of 41 references.

References · 17

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  4. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  10. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  11. 2017Progressive Neural Architecture Searchcited by 483
  12. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
5 more
  1. 2019Human uncertainty makes classification more robustcited by 231
  2. 2020Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistencycited by 77
  3. 2019Recurrence is required to capture the representational dynamics of the human visual systemcited by 58
  4. 2021People construct simplified mental representations to plancited by 45
  5. 2019A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNscited by 9

Built on by

Not cited yet.