# Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network

arXiv: https://arxiv.org/abs/2001.06268 · HTML: https://mlsanity.com/paper/2001.06268/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2020.

## Built on by (most cited first, 24 of 24)

- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [Bottleneck Transformers for Visual Recognition](https://mlsanity.com/paper/2101.11605/) (2021)
- [Revisiting ResNets: Improved Training and Scaling Strategies](https://mlsanity.com/paper/2103.07579/) (2021)
- [LambdaNetworks: Modeling Long-Range Interactions Without Attention](https://mlsanity.com/paper/2102.08602/) (2021)
- [On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness](https://mlsanity.com/paper/2102.11273/) (2021)
- [AugMax: Adversarial Composition of Random Augmentations for Robust Training](https://mlsanity.com/paper/2110.13771/) (2021)
- [A linearized framework and a new benchmark for model selection for fine-tuning](https://mlsanity.com/paper/2102.00084/) (2021)
- [Defending Against Image Corruptions Through Adversarial Augmentations](https://mlsanity.com/paper/2104.01086/) (2021)
- [Network Augmentation for Tiny Deep Learning](https://mlsanity.com/paper/2110.08890/) (2021)
- [How Does Frequency Bias Affect the Robustness of Neural Image Classifiers against Common Corruption and Adversarial Perturbations?](https://mlsanity.com/paper/2205.04533/) (2022)
- [Solving ImageNet: a Unified Scheme for Training any Backbone to Top Results](https://mlsanity.com/paper/2204.03475/) (2022)
- [UFO: Unified Feature Optimization](https://mlsanity.com/paper/2207.10341/) (2022)
- [Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training](https://mlsanity.com/paper/2106.03640/) (2021)
- [Improving Model Generalization by Agreement of Learned Representations from Data Augmentation](https://mlsanity.com/paper/2110.10536/) (2021)
- [Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs](https://mlsanity.com/paper/2110.10645/) (2021)
- [Frequency Dropout: Feature-Level Regularization via Randomized Filtering](https://mlsanity.com/paper/2209.09844/) (2022)
- [Robust Classification by Coupling Data Mollification with Label Smoothing](https://mlsanity.com/paper/2406.01494/) (2024)
- [NetBooster: Empowering Tiny Deep Learning By Standing on the Shoulders of Deep Giants](https://mlsanity.com/paper/2306.13586/) (2023)
- [Sequential Random Network for Fine-grained Image Classification](https://mlsanity.com/paper/2103.07230/) (2021)
- [Improving generalization with synthetic training data for deep learning based quality inspection](https://mlsanity.com/paper/2202.12818/) (2022)
- [Pixel to Binary Embedding Towards Robustness for CNNs](https://mlsanity.com/paper/2206.05898/) (2022)
- [NasHD: Efficient ViT Architecture Performance Ranking using Hyperdimensional Computing](https://mlsanity.com/paper/2209.11356/) (2022)
- [Explainability and Robustness of Deep Visual Classification Models](https://mlsanity.com/paper/2301.01343/) (2023)

## Abstract (arXiv)

Recent studies in image classification have demonstrated a variety of techniques for improving the performance of Convolutional Neural Networks (CNNs). However, attempts to combine existing techniques to create a practical model are still uncommon. In this study, we carry out extensive experiments to validate that carefully assembling these techniques and applying them to basic CNN models (e.g. ResNet and MobileNet) can improve the accuracy and robustness of the models while minimizing the loss of throughput. Our proposed assembled ResNet-50 shows improvements in top-1 accuracy from 76.3% to 82.78%, mCE from 76.0% to 48.9% and mFR from 57.7% to 32.3% on ILSVRC2012 validation set. With these improvements, inference throughput only decreases from 536 to 312. To verify the performance improvement in transfer learning, fine grained classification and image retrieval tasks were tested on several public datasets and showed that the improvement to backbone network performance boosted transfer learning performance significantly. Our approach achieved 1st place in the iFood Competition Fine-Grained Visual Recognition at CVPR 2019, and the source code and trained models are available at https://github.com/clovaai/assembled-cnn