# Covariant spatio-temporal receptive fields for spiking neural networks

arXiv: https://arxiv.org/abs/2405.00318 · HTML: https://mlsanity.com/paper/2405.00318/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 89 references that are in the map.
- First posted on arXiv 1 May 2024.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [B-Spline CNNs on Lie Groups](https://mlsanity.com/paper/1909.12057/) (2019)
- [HFirst: A Temporal Approach to Object Recognition](https://mlsanity.com/paper/1508.01176/) (2015)
- [Structured Receptive Fields in CNNs](https://mlsanity.com/paper/1605.02971/) (2016)
- [Scale-covariant and scale-invariant Gaussian derivative networks](https://mlsanity.com/paper/2011.14759/) (2020)
- [Neuromorphic Intermediate Representation: A Unified Instruction Set for Interoperable Brain-Inspired Computing](https://mlsanity.com/paper/2311.14641/) (2023)
- [Normative theory of visual receptive fields](https://mlsanity.com/paper/1701.06333/) (2017)
- [DOTIE - Detecting Objects through Temporal Isolation of Events using a Spiking Architecture](https://mlsanity.com/paper/2210.00975/) (2022)
- [Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges](https://mlsanity.com/paper/2004.01536/) (2020)
- [A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time](https://mlsanity.com/paper/2202.09209/) (2022)
- [Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2303.09803/) (2023)
- [Time-causal and time-recursive spatio-temporal receptive fields](https://mlsanity.com/paper/1504.02648/) (2015)
- [Discrete approximations of Gaussian smoothing and Gaussian derivatives](https://mlsanity.com/paper/2311.11317/) (2023)
- [A deep convolutional neural network that is invariant to time rescaling](https://mlsanity.com/paper/2107.04616/) (2021)
- [Natural-gradient learning for spiking neurons](https://mlsanity.com/paper/2011.11710/) (2020)
- [Dynamic texture recognition using time-causal and time-recursive spatio-temporal receptive fields](https://mlsanity.com/paper/1710.04842/) (2017)
- [Temporal scale selection in time-causal scale space](https://mlsanity.com/paper/1701.05088/) (2017)
- [Learning temporal relationships between symbols with Laplace Neural Manifolds](https://mlsanity.com/paper/2302.10163/) (2023)
- [Spiking Neural Network Nonlinear Demapping on Neuromorphic Hardware for IM/DD Optical Communication](https://mlsanity.com/paper/2302.14726/) (2023)

## Built on by (most cited first, 3 of 3)

- [Flow Equivariant Recurrent Neural Networks](https://mlsanity.com/paper/2507.14793/) (2025)
- [GERD: Geometric event response data generation](https://mlsanity.com/paper/2412.03259/) (2024)
- [Scale-covariant spiking wavelets](https://mlsanity.com/paper/2602.02020/) (2026)

## Abstract (arXiv)

Biological nervous systems constitute important sources of inspiration towards computers that are faster, cheaper, and more energy efficient. Neuromorphic disciplines view the brain as a coevolved system, simultaneously optimizing the hardware and the algorithms running on it. There are clear efficiency gains when bringing the computations into a physical substrate, but we presently lack theories to guide efficient implementations. Here, we present a principled computational model for neuromorphic systems in terms of spatio-temporal receptive fields, based on affine Gaussian kernels over space and leaky-integrator and leaky integrate-and-fire models over time. Our theory is provably covariant to spatial affine and temporal scaling transformations, and with close similarities to the visual processing in mammalian brains. We use these spatio-temporal receptive fields as a prior in an event-based vision task, and show that this improves the training of spiking networks, which otherwise is known as problematic for event-based vision. This work combines efforts within scale-space theory and computational neuroscience to identify theoretically well-founded ways to process spatio-temporal signals in neuromorphic systems. Our contributions are immediately relevant for signal processing and event-based vision, and can be extended to other processing tasks over space and time, such as memory and control.