# Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields

arXiv: https://arxiv.org/abs/2303.09803 · HTML: https://mlsanity.com/paper/2303.09803/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 89 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 15 of 15)

- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)
- [B-Spline CNNs on Lie Groups](https://mlsanity.com/paper/1909.12057/) (2019)
- [Understanding Deep Convolutional Networks](https://mlsanity.com/paper/1601.04920/) (2016)
- [DISCO: accurate Discrete Scale Convolutions](https://mlsanity.com/paper/2106.02733/) (2021)
- [Structured Receptive Fields in CNNs](https://mlsanity.com/paper/1605.02971/) (2016)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [Scale-covariant and scale-invariant Gaussian derivative networks](https://mlsanity.com/paper/2011.14759/) (2020)
- [Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales](https://mlsanity.com/paper/2106.06418/) (2021)
- [Normative theory of visual receptive fields](https://mlsanity.com/paper/1701.06333/) (2017)
- [Scale Equivariant U-Net](https://mlsanity.com/paper/2210.04508/) (2022)
- [Fully trainable Gaussian derivative convolutional layer](https://mlsanity.com/paper/2207.08424/) (2022)
- [Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade](https://mlsanity.com/paper/1905.13555/) (2019)
- [A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time](https://mlsanity.com/paper/2202.09209/) (2022)
- [Time-causal and time-recursive spatio-temporal receptive fields](https://mlsanity.com/paper/1504.02648/) (2015)

## Built on by (most cited first, 8 of 8)

- [Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2311.10543/) (2023)
- [Covariant spatio-temporal receptive fields for spiking neural networks](https://mlsanity.com/paper/2405.00318/) (2024)
- [The Master Key Filters Hypothesis: Deep Filters Are General](https://mlsanity.com/paper/2412.16751/) (2024)
- [Relationships between the degrees of freedom in the affine Gaussian derivative model for visual receptive fields and 2-D affine image transformations, with application to covariance properties of simple cells in the primary visual cortex](https://mlsanity.com/paper/2411.05673/) (2024)
- [The Quest for Universal Master Key Filters in DS-CNNs](https://mlsanity.com/paper/2509.11711/) (2025)
- [GERD: Geometric event response data generation](https://mlsanity.com/paper/2412.03259/) (2024)
- [Hybrid Lie semi-group and cascade structures for the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2509.15748/) (2025)
- [Scale-invariant Gaussian derivative residual networks](https://mlsanity.com/paper/2603.02843/) (2026)

## Abstract (arXiv)

This paper presents a theory for how geometric image transformations can be handled by a first layer of linear receptive fields, in terms of true covariance properties, which, in turn, enable geometric invariance properties at higher levels in the visual hierarchy. Specifically, we develop this theory for a generalized Gaussian derivative model for visual receptive fields, which is derived in an axiomatic manner from first principles, that reflect symmetry properties of the environment, complemented by structural assumptions to guarantee internally consistent treatment of image structures over multiple spatio-temporal scales. It is shown how the studied generalized Gaussian derivative model for visual receptive fields obeys true covariance properties under spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations, implying that a vision system, based on image and video measurements in terms of the receptive fields according to this model, can to first order of approximation handle the image and video deformations between multiple views of objects delimited by smooth surfaces, as well as between multiple views of spatio-temporal events, under varying relative motions between the objects and events in the world and the observer. We conclude by describing implications of the presented theory for biological vision, regarding connections between the variabilities of the shapes of biological visual receptive fields and the variabilities of spatial and spatio-temporal image structures under natural image transformations.