# Just a Matter of Scale? Reevaluating Scale Equivariance in Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2211.10288 · HTML: https://mlsanity.com/paper/2211.10288/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 18 Nov 2022.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)
- [A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection](https://mlsanity.com/paper/1607.07155/) (2016)
- [CrowdNet: A Deep Convolutional Network for Dense Crowd Counting](https://mlsanity.com/paper/1608.06197/) (2016)
- [The Mapillary Traffic Sign Dataset for Detection and Classification on a Global Scale](https://mlsanity.com/paper/1909.04422/) (2019)
- [Scale-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/1411.6369/) (2014)
- [DISCO: accurate Discrete Scale Convolutions](https://mlsanity.com/paper/2106.02733/) (2021)
- [Locally Scale-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/1412.5104/) (2014)
- [Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/1906.03861/) (2019)
- [Learning scale-variant and scale-invariant features for deep image classification](https://mlsanity.com/paper/1602.01255/) (2016)
- [Scale equivariance in CNNs with vector fields](https://mlsanity.com/paper/1807.11783/) (2018)
- [Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters](https://mlsanity.com/paper/1909.11193/) (2019)
- [Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales](https://mlsanity.com/paper/2106.06418/) (2021)
- [Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges](https://mlsanity.com/paper/2004.01536/) (2020)
- [Stride and Translation Invariance in CNNs](https://mlsanity.com/paper/2103.10097/) (2021)
- [Understanding when spatial transformer networks do not support invariance, and what to do about it](https://mlsanity.com/paper/2004.11678/) (2020)

## Built on by (most cited first, 2 of 2)

- [What Affects Learned Equivariance in Deep Image Recognition Models?](https://mlsanity.com/paper/2304.02628/) (2023)
- [Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations](https://mlsanity.com/paper/2409.11140/) (2024)

## Abstract (arXiv)

The widespread success of convolutional neural networks may largely be attributed to their intrinsic property of translation equivariance. However, convolutions are not equivariant to variations in scale and fail to generalize to objects of different sizes. Despite recent advances in this field, it remains unclear how well current methods generalize to unobserved scales on real-world data and to what extent scale equivariance plays a role. To address this, we propose the novel Scaled and Translated Image Recognition (STIR) benchmark based on four different domains. Additionally, we introduce a new family of models that applies many re-scaled kernels with shared weights in parallel and then selects the most appropriate one. Our experimental results on STIR show that both the existing and proposed approaches can improve generalization across scales compared to standard convolutions. We also demonstrate that our family of models is able to generalize well towards larger scales and improve scale equivariance. Moreover, due to their unique design we can validate that kernel selection is consistent with input scale. Even so, none of the evaluated models maintain their performance for large differences in scale, demonstrating that a general understanding of how scale equivariance can improve generalization and robustness is still lacking.