Merging of neural networks
Not on the map. Why?
Abstract (arXiv)
We propose a simple scheme for merging two neural networks trained with different starting initialization into a single one with the same size as the original ones. We do this by carefully selecting channels from each input network. Our procedure might be used as a finalization step after one tries multiple starting seeds to avoid an unlucky one. We also show that training two networks and merging them leads to better performance than training a single network for an extended period of time. Availability: https://github.com/fmfi-compbio/neural-network-merging
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 21 Apr 2022.
Builds on
14 of 24 references.
References · 14
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
- 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
- 2020Federated Learning with Matched Averagingcited by 564
- 2019Importance Estimation for Neural Network Pruningcited by 541
- 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 442
- 2021ResNet strikes back: An improved training procedure in timmcited by 368
- 2021Knowledge distillation: A good teacher is patient and consistentcited by 259
- 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
- 2021Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer visioncited by 81
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.