# When do Convolutional Neural Networks Stop Learning?

arXiv: https://arxiv.org/abs/2403.02473 · HTML: https://mlsanity.com/paper/2403.02473/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?](https://mlsanity.com/paper/1706.00712/) (2017)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [CondenseNet: An Efficient DenseNet using Learned Group Convolutions](https://mlsanity.com/paper/1711.09224/) (2017)
- [Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping](https://mlsanity.com/paper/2010.13369/) (2020)
- [ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks](https://mlsanity.com/paper/1811.10495/) (2018)
- [Early Stopping without a Validation Set](https://mlsanity.com/paper/1703.09580/) (2017)
- [DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring](https://mlsanity.com/paper/2103.09962/) (2021)
- [Biologically Inspired Mechanisms for Adversarial Robustness](https://mlsanity.com/paper/2006.16427/) (2020)
- [Going Deep in Medical Image Analysis: Concepts, Methods, Challenges and Future Directions](https://mlsanity.com/paper/1902.05655/) (2019)
- [Neuron Merging: Compensating for Pruned Neurons](https://mlsanity.com/paper/2010.13160/) (2020)
- [Early Stopping is Nonparametric Variational Inference](https://mlsanity.com/paper/1504.01344/) (2015)
- [Certifying Strategyproof Auction Networks](https://mlsanity.com/paper/2006.08742/) (2020)
- [PyGlove: Symbolic Programming for Automated Machine Learning](https://mlsanity.com/paper/2101.08809/) (2021)
- [AViD Dataset: Anonymized Videos from Diverse Countries](https://mlsanity.com/paper/2007.05515/) (2020)
- [Adam with Bandit Sampling for Deep Learning](https://mlsanity.com/paper/2010.12986/) (2020)
- [Channel-Wise Early Stopping without a Validation Set via NNK Polytope Interpolation](https://mlsanity.com/paper/2107.12972/) (2021)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) have demonstrated outstanding performance in computer vision tasks such as image classification, detection, segmentation, and medical image analysis. In general, an arbitrary number of epochs is used to train such neural networks. In a single epoch, the entire training data -- divided by batch size -- are fed to the network. In practice, validation error with training loss is used to estimate the neural network's generalization, which indicates the optimal learning capacity of the network. Current practice is to stop training when the training loss decreases and the gap between training and validation error increases (i.e., the generalization gap) to avoid overfitting. However, this is a trial-and-error-based approach which raises a critical question: Is it possible to estimate when neural networks stop learning based on training data? This research work introduces a hypothesis that analyzes the data variation across all the layers of a CNN variant to anticipate its near-optimal learning capacity. In the training phase, we use our hypothesis to anticipate the near-optimal learning capacity of a CNN variant without using any validation data. Our hypothesis can be deployed as a plug-and-play to any existing CNN variant without introducing additional trainable parameters to the network. We test our hypothesis on six different CNN variants and three different general image datasets (CIFAR10, CIFAR100, and SVHN). The result based on these CNN variants and datasets shows that our hypothesis saves 58.49% of computational time (on average) in training. We further conduct our hypothesis on ten medical image datasets and compared with the MedMNIST-V2 benchmark. Based on our experimental result, we save ≈ 44.1% of computational time without losing accuracy against the MedMNIST-V2 benchmark.