# Incremental Training of Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1803.10232 · HTML: https://mlsanity.com/paper/1803.10232/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2018.

## Built on by (most cited first, 3 of 3)

- [Active learning for data streams: a survey](https://mlsanity.com/paper/2302.08893/) (2023)
- [DyRep: Bootstrapping Training with Dynamic Re-parameterization](https://mlsanity.com/paper/2203.12868/) (2022)
- [When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks](https://mlsanity.com/paper/2401.03104/) (2024)

## Abstract (arXiv)

We propose an incremental training method that partitions the original network into sub-networks, which are then gradually incorporated in the running network during the training process. To allow for a smooth dynamic growth of the network, we introduce a look-ahead initialization that outperforms the random initialization. We demonstrate that our incremental approach reaches the reference network baseline accuracy. Additionally, it allows to identify smaller partitions of the original state-of-the-art network, that deliver the same final accuracy, by using only a fraction of the global number of parameters. This allows for a potential speedup of the training time of several factors. We report training results on CIFAR-10 for ResNet and VGGNet.