# Growing Neural Network with Shared Parameter

arXiv: https://arxiv.org/abs/2201.06500 · HTML: https://mlsanity.com/paper/2201.06500/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 7 of 7)

- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [Net2Net: Accelerating Learning via Knowledge Transfer](https://mlsanity.com/paper/1511.05641/) (2015)
- [Splitting Steepest Descent for Growing Neural Architectures](https://mlsanity.com/paper/1910.02366/) (2019)
- [Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks](https://mlsanity.com/paper/2102.08574/) (2021)
- [Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent](https://mlsanity.com/paper/1910.03103/) (2019)
- [Efficient Forward Architecture Search](https://mlsanity.com/paper/1905.13360/) (2019)

## Abstract (arXiv)

We propose a general method for growing neural network with shared parameter by matching trained network to new input. By leveraging Hoeffding's inequality, we provide a theoretical base for improving performance by adding subnetwork to existing network. With the theoretical base of adding new subnetwork, we implement a matching method to apply trained subnetwork of existing network to new input. Our method has shown the ability to improve performance with higher parameter efficiency. It can also be applied to trans-task case and realize transfer learning by changing the combination of subnetworks without training on new task.