# 2D versus 3D Convolutional Spiking Neural Networks Trained with Unsupervised STDP for Human Action Recognition

arXiv: https://arxiv.org/abs/2205.13474 · HTML: https://mlsanity.com/paper/2205.13474/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [SpikeMS: Deep Spiking Neural Network for Motion Segmentation](https://mlsanity.com/paper/2105.06562/) (2021)
- [SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron](https://mlsanity.com/paper/1903.02440/) (2019)
- [Tuning Convolutional Spiking Neural Network with Biologically-plausible Reward Propagation](https://mlsanity.com/paper/2010.04434/) (2020)
- [Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP](https://mlsanity.com/paper/1904.01908/) (2019)
- [A Study On the Effects of Pre-processing On Spatio-temporal Action Recognition Using Spiking Neural Networks Trained with STDP](https://mlsanity.com/paper/2105.14740/) (2021)

## Built on by (most cited first, 5 of 5)

- [Neuronal Competition Groups with Supervised STDP for Spike-Based Classification](https://mlsanity.com/paper/2410.17066/) (2024)
- [Spiking Two-Stream Methods with Unsupervised STDP-based Learning for Action Recognition](https://mlsanity.com/paper/2306.13783/) (2023)
- [S3TC: Spiking Separated Spatial and Temporal Convolutions with Unsupervised STDP-based Learning for Action Recognition](https://mlsanity.com/paper/2309.12761/) (2023)
- [VT-LVLM-AR: A Video-Temporal Large Vision-Language Model Adapter for Fine-Grained Action Recognition in Long-Term Videos](https://mlsanity.com/paper/2508.15903/) (2025)
- [Investigating Continuous Learning in Spiking Neural Networks](https://mlsanity.com/paper/2310.05343/) (2023)

## Abstract (arXiv)

Current advances in technology have highlighted the importance of video analysis in the domain of computer vision. However, video analysis has considerably high computational costs with traditional artificial neural networks (ANNs). Spiking neural networks (SNNs) are third generation biologically plausible models that process the information in the form of spikes. Unsupervised learning with SNNs using the spike timing dependent plasticity (STDP) rule has the potential to overcome some bottlenecks of regular artificial neural networks, but STDP-based SNNs are still immature and their performance is far behind that of ANNs. In this work, we study the performance of SNNs when challenged with the task of human action recognition, because this task has many real-time applications in computer vision, such as video surveillance. In this paper we introduce a multi-layered 3D convolutional SNN model trained with unsupervised STDP. We compare the performance of this model to those of a 2D STDP-based SNN when challenged with the KTH and Weizmann datasets. We also compare single-layer and multi-layer versions of these models in order to get an accurate assessment of their performance. We show that STDP-based convolutional SNNs can learn motion patterns using 3D kernels, thus enabling motion-based recognition from videos. Finally, we give evidence that 3D convolution is superior to 2D convolution with STDP-based SNNs, especially when dealing with long video sequences.