# Should You Go Deeper? Optimizing Convolutional Neural Network Architectures without Training by Receptive Field Analysis

arXiv: https://arxiv.org/abs/2106.12307 · HTML: https://mlsanity.com/paper/2106.12307/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Highway Networks](https://mlsanity.com/paper/1505.00387/) (2015)
- [Feature Space Saturation during Training](https://mlsanity.com/paper/2006.08679/) (2020)

## Built on by (most cited first, 7 of 7)

- [Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.00637/) (2023)
- [Receptive Field Refinement for Convolutional Neural Networks Reliably Improves Predictive Performance](https://mlsanity.com/paper/2211.14487/) (2022)
- [AI-in-the-Loop -- The impact of HMI in AI-based Application](https://mlsanity.com/paper/2303.11508/) (2023)
- [CAMS: Convolution and Attention-Free Mamba-based Cardiac Image Segmentation](https://mlsanity.com/paper/2406.05786/) (2024)
- [AI for Closed-Loop Control Systems -- New Opportunities for Modeling, Designing, and Tuning Control Systems](https://mlsanity.com/paper/2201.06961/) (2022)
- [FIND-Net -- Fourier-Integrated Network with Dictionary Kernels for Metal Artifact Reduction](https://mlsanity.com/paper/2508.10617/) (2025)
- [Your Image is My Video: Reshaping the Receptive Field via Image-To-Video Differentiable AutoAugmentation and Fusion](https://mlsanity.com/paper/2403.15194/) (2024)

## Abstract (arXiv)

When optimizing convolutional neural networks (CNN) for a specific image-based task, specialists commonly overshoot the number of convolutional layers in their designs. By implication, these CNNs are unnecessarily resource intensive to train and deploy, with diminishing beneficial effects on the predictive performance. The features a convolutional layer can process are strictly limited by its receptive field. By layer-wise analyzing the size of the receptive fields, we can reliably predict sequences of layers that will not contribute qualitatively to the test accuracy in the given CNN architecture. Based on this analysis, we propose design strategies based on a so-called border layer. This layer allows to identify unproductive convolutional layers and hence to resolve these inefficiencies, optimize the explainability and the computational performance of CNNs. Since neither the strategies nor the analysis requires training of the actual model, these insights allow for a very efficient design process of CNN architectures, which might be automated in the future.