# Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations

arXiv: https://arxiv.org/abs/2409.11140 · HTML: https://mlsanity.com/paper/2409.11140/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 88 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 30 of 47)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution](https://mlsanity.com/paper/1904.05049/) (2019)
- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)
- [B-Spline CNNs on Lie Groups](https://mlsanity.com/paper/1909.12057/) (2019)
- [Geometric Deep Learning and Equivariant Neural Networks](https://mlsanity.com/paper/2105.13926/) (2021)
- [Attribution in Scale and Space](https://mlsanity.com/paper/2004.03383/) (2020)
- [Gabor Convolutional Networks](https://mlsanity.com/paper/1705.01450/) (2017)
- [Every Model Learned by Gradient Descent Is Approximately a Kernel Machine](https://mlsanity.com/paper/2012.00152/) (2020)
- [The Mapillary Traffic Sign Dataset for Detection and Classification on a Global Scale](https://mlsanity.com/paper/1909.04422/) (2019)
- [Scale-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/1411.6369/) (2014)
- [DISCO: accurate Discrete Scale Convolutions](https://mlsanity.com/paper/2106.02733/) (2021)
- [Locally Scale-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/1412.5104/) (2014)
- [PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions](https://mlsanity.com/paper/2007.10408/) (2020)
- [Structured Receptive Fields in CNNs](https://mlsanity.com/paper/1605.02971/) (2016)
- [PDE-based Group Equivariant Convolutional Neural Networks](https://mlsanity.com/paper/2001.09046/) (2020)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [Scale-Equalizing Pyramid Convolution for Object Detection](https://mlsanity.com/paper/2005.03101/) (2020)
- [Scale-covariant and scale-invariant Gaussian derivative networks](https://mlsanity.com/paper/2011.14759/) (2020)
- [Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/1906.03861/) (2019)
- [Scale equivariance in CNNs with vector fields](https://mlsanity.com/paper/1807.11783/) (2018)
- [Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales](https://mlsanity.com/paper/2106.06418/) (2021)
- [Normative theory of visual receptive fields](https://mlsanity.com/paper/1701.06333/) (2017)
- [Scale Equivariant U-Net](https://mlsanity.com/paper/2210.04508/) (2022)
- [Blurring the Line Between Structure and Learning to Optimize and Adapt Receptive Fields](https://mlsanity.com/paper/1904.11487/) (2019)

## Built on by (most cited first, 3 of 3)

- [Modelling and analysis of the 8 filters from the "master key filters hypothesis" for depthwise-separable deep networks in relation to idealized receptive fields based on scale-space theory](https://mlsanity.com/paper/2509.12746/) (2025)
- [Hybrid Lie semi-group and cascade structures for the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2509.15748/) (2025)
- [Scale-invariant Gaussian derivative residual networks](https://mlsanity.com/paper/2603.02843/) (2026)

## Abstract (arXiv)

This paper presents an in-depth analysis of the scale generalisation properties of the scale-covariant and scale-invariant Gaussian derivative networks, complemented with both conceptual and algorithmic extensions. For this purpose, Gaussian derivative networks (GaussDerNets) are evaluated on new rescaled versions of the Fashion-MNIST and the CIFAR-10 datasets, with spatial scaling variations over a factor of 4 in the testing data, that are not present in the training data. Additionally, evaluations on the previously existing STIR datasets show that the GaussDerNets achieve better scale generalisation than previously reported for these datasets for other types of deep networks. We first experimentally demonstrate that the GaussDerNets have quite good scale generalisation properties on the new datasets, and that average pooling of feature responses over scales may sometimes also lead to better results than the previously used approach of max pooling over scales. Then, we demonstrate that using a spatial max pooling mechanism after the final layer enables localisation of non-centred objects in image domain, with maintained scale generalisation properties. We also show that regularisation during training, by applying dropout across the scale channels, referred to as scale-channel dropout, improves both the performance and the scale generalisation. In additional ablation studies, we demonstrate that discretisations of GaussDerNets, based on the discrete analogue of the Gaussian kernel in combination with central difference operators, perform best or among the best, compared to a set of other discrete approximations of the Gaussian derivative kernels. Finally, by visualising the activation maps and the learned receptive fields, we demonstrate that the GaussDerNets have very good explainability properties.