# Scale Equivariant U-Net

arXiv: https://arxiv.org/abs/2210.04508 · HTML: https://mlsanity.com/paper/2210.04508/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Learning Steerable Filters for Rotation Equivariant CNNs](https://mlsanity.com/paper/1711.07289/) (2017)
- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)
- [DISCO: accurate Discrete Scale Convolutions](https://mlsanity.com/paper/2106.02733/) (2021)
- [Scale-covariant and scale-invariant Gaussian derivative networks](https://mlsanity.com/paper/2011.14759/) (2020)
- [Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/1906.03861/) (2019)
- [Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges](https://mlsanity.com/paper/2004.01536/) (2020)
- [Scale Equivariant Neural Networks with Morphological Scale-Spaces](https://mlsanity.com/paper/2105.01335/) (2021)

## Built on by (most cited first, 13 of 13)

- [What Affects Learned Equivariance in Deep Image Recognition Models?](https://mlsanity.com/paper/2304.02628/) (2023)
- [Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2303.09803/) (2023)
- [Discrete approximations of Gaussian smoothing and Gaussian derivatives](https://mlsanity.com/paper/2311.11317/) (2023)
- [HyperSpace: Hypernetworks for spacing-adaptive image segmentation](https://mlsanity.com/paper/2407.03681/) (2024)
- [Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2311.10543/) (2023)
- [Riesz feature representation: scale equivariant scattering network for classification tasks](https://mlsanity.com/paper/2307.08467/) (2023)
- [Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators](https://mlsanity.com/paper/2405.05095/) (2024)
- [Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations](https://mlsanity.com/paper/2409.11140/) (2024)
- [Current Symmetry Group Equivariant Convolution Frameworks for Representation Learning](https://mlsanity.com/paper/2409.07327/) (2024)
- [Geometric Generative Models based on Morphological Equivariant PDEs and GANs](https://mlsanity.com/paper/2403.14897/) (2024)
- [Modelling and analysis of the 8 filters from the "master key filters hypothesis" for depthwise-separable deep networks in relation to idealized receptive fields based on scale-space theory](https://mlsanity.com/paper/2509.12746/) (2025)
- [Hybrid Lie semi-group and cascade structures for the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2509.15748/) (2025)
- [Scale-invariant Gaussian derivative residual networks](https://mlsanity.com/paper/2603.02843/) (2026)

## Abstract (arXiv)

In neural networks, the property of being equivariant to transformations improves generalization when the corresponding symmetry is present in the data. In particular, scale-equivariant networks are suited to computer vision tasks where the same classes of objects appear at different scales, like in most semantic segmentation tasks. Recently, convolutional layers equivariant to a semigroup of scalings and translations have been proposed. However, the equivariance of subsampling and upsampling has never been explicitly studied even though they are necessary building blocks in some segmentation architectures. The U-Net is a representative example of such architectures, which includes the basic elements used for state-of-the-art semantic segmentation. Therefore, this paper introduces the Scale Equivariant U-Net (SEU-Net), a U-Net that is made approximately equivariant to a semigroup of scales and translations through careful application of subsampling and upsampling layers and the use of aforementioned scale-equivariant layers. Moreover, a scale-dropout is proposed in order to improve generalization to different scales in approximately scale-equivariant architectures. The proposed SEU-Net is trained for semantic segmentation of the Oxford Pet IIIT and the DIC-C2DH-HeLa dataset for cell segmentation. The generalization metric to unseen scales is dramatically improved in comparison to the U-Net, even when the U-Net is trained with scale jittering, and to a scale-equivariant architecture that does not perform upsampling operators inside the equivariant pipeline. The scale-dropout induces better generalization on the scale-equivariant models in the Pet experiment, but not on the cell segmentation experiment.