# Elastic Neural Networks for Classification

arXiv: https://arxiv.org/abs/1810.00589 · HTML: https://mlsanity.com/paper/1810.00589/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2018.

## Built on by (most cited first, 5 of 5)

- [Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inference](https://mlsanity.com/paper/2312.03464/) (2023)
- [HashReID: Dynamic Network with Binary Codes for Efficient Person Re-identification](https://mlsanity.com/paper/2308.11900/) (2023)
- [QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architectures](https://mlsanity.com/paper/2212.12866/) (2022)
- [ItNet: iterative neural networks with small graphs for accurate, efficient and anytime semantic segmentation](https://mlsanity.com/paper/2101.08685/) (2021)
- [using multiple losses for accurate facial age estimation](https://mlsanity.com/paper/2106.09393/) (2021)

## Abstract (arXiv)

In this work we propose a framework for improving the performance of any deep neural network that may suffer from vanishing gradients. To address the vanishing gradient issue, we study a framework, where we insert an intermediate output branch after each layer in the computational graph and use the corresponding prediction loss for feeding the gradient to the early layers. The framework - which we name Elastic network - is tested with several well-known networks on CIFAR10 and CIFAR100 datasets, and the experimental results show that the proposed framework improves the accuracy on both shallow networks (e.g., MobileNet) and deep convolutional neural networks (e.g., DenseNet). We also identify the types of networks where the framework does not improve the performance and discuss the reasons. Finally, as a side product, the computational complexity of the resulting networks can be adjusted in an elastic manner by selecting the output branch according to current computational budget.