# Learn Like The Pro: Norms from Theory to Size Neural Computation

arXiv: https://arxiv.org/abs/2106.11409 · HTML: https://mlsanity.com/paper/2106.11409/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 39 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Informative Neural Ensemble Kalman Learning](https://mlsanity.com/paper/2008.09915/) (2020)
- [Neural Integration of Continuous Dynamics](https://mlsanity.com/paper/1911.10309/) (2019)
- [Neural Networks and Chaos: Construction, Evaluation of Chaotic Networks, and Prediction of Chaos with Multilayer Feedforward Networks](https://mlsanity.com/paper/1608.05916/) (2016)
- [A Practical Approach to Sizing Neural Networks](https://mlsanity.com/paper/1810.02328/) (2018)

## Built on by (most cited first, 1 of 1)

- [Multirotor Ensemble Model Predictive Control I: Simulation Experiments](https://mlsanity.com/paper/2305.12625/) (2023)

## Abstract (arXiv)

The optimal design of neural networks is a critical problem in many applications. Here, we investigate how dynamical systems with polynomial nonlinearities can inform the design of neural systems that seek to emulate them. We propose a Learnability metric and its associated features to quantify the near-equilibrium behavior of learning dynamics. Equating the Learnability of neural systems with equivalent parameter estimation metric of the reference system establishes bounds on network structure. In this way, norms from theory provide a good first guess for neural structure, which may then further adapt with data. The proposed approach neither requires training nor training data. It reveals exact sizing for a class of neural networks with multiplicative nodes that mimic continuous- or discrete-time polynomial dynamics. It also provides relatively tight lower size bounds for classical feed-forward networks that is consistent with simulated assessments.