Training of deep residual networks with stochastic MG/OPT
Not on the map. Why?
Abstract (arXiv)
We train deep residual networks with a stochastic variant of the nonlinear multigrid method MG/OPT. To build the multilevel hierarchy, we use the dynamical systems viewpoint specific to residual networks. We report significant speed-ups and additional robustness for training MNIST on deep residual networks. Our numerical experiments also indicate that multilevel training can be used as a pruning technique, as many of the auxiliary networks have accuracies comparable to the original network.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 11 of its 25 references that are in the map. First posted on arXiv 9 Aug 2021.
Builds on
11 of 25 references.
References · 11
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
- 2017Stable Architectures for Deep Neural Networkscited by 341
- 2017Multi-level Residual Networks from Dynamical Systems Viewcited by 59
- 2020Multilevel Minimization for Deep Residual Networkscited by 11
- 2016DSD: Dense-Sparse-Dense Training for Deep Neural Networkscited by 7
- 2019Multilevel Initialization for Layer-Parallel Deep Neural Network Trainingcited by 7
- 2017The Power of Sparsity in Convolutional Neural Networkscited by 2
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.