# Learning from Event Cameras with Sparse Spiking Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2104.12579 · HTML: https://mlsanity.com/paper/2104.12579/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [Event-based Asynchronous Sparse Convolutional Networks](https://mlsanity.com/paper/2003.09148/) (2020)
- [Low-activity supervised convolutional spiking neural networks applied to speech commands recognition](https://mlsanity.com/paper/2011.06846/) (2020)
- [Design Space Exploration of Hardware Spiking Neurons for Embedded Artificial Intelligence](https://mlsanity.com/paper/1910.01010/) (2019)

## Built on by (most cited first, 25 of 25)

- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Object Detection with Spiking Neural Networks on Automotive Event Data](https://mlsanity.com/paper/2205.04339/) (2022)
- [State Space Models for Event Cameras](https://mlsanity.com/paper/2402.15584/) (2024)
- [Optical flow estimation from event-based cameras and spiking neural networks](https://mlsanity.com/paper/2302.06492/) (2023)
- [Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models](https://mlsanity.com/paper/2404.18508/) (2024)
- [V2CE: Video to Continuous Events Simulator](https://mlsanity.com/paper/2309.08891/) (2023)
- [Embedded Graph Convolutional Networks for Real-Time Event Data Processing on SoC FPGAs](https://mlsanity.com/paper/2406.07318/) (2024)
- [Finding Meaning in Points: Weakly Supervised Semantic Segmentation for Event Cameras](https://mlsanity.com/paper/2407.11216/) (2024)
- [MK-SGN: A Spiking Graph Convolutional Network with Multimodal Fusion and Knowledge Distillation for Skeleton-based Action Recognition](https://mlsanity.com/paper/2404.10210/) (2024)
- [Navigating Local Minima in Quantized Spiking Neural Networks](https://mlsanity.com/paper/2202.07221/) (2022)
- [Time-Resolved MNIST Dataset for Single-Photon Recognition](https://mlsanity.com/paper/2410.16744/) (2024)
- [Spiking GATs: Learning Graph Attentions via Spiking Neural Network](https://mlsanity.com/paper/2209.13539/) (2022)
- [Event Trojan: Asynchronous Event-based Backdoor Attacks](https://mlsanity.com/paper/2407.06838/) (2024)
- [Evaluating Image-Based Face and Eye Tracking with Event Cameras](https://mlsanity.com/paper/2408.10395/) (2024)
- [Spiking monocular event based 6D pose estimation for space application](https://mlsanity.com/paper/2501.02916/) (2025)
- [NU-AIR -- A Neuromorphic Urban Aerial Dataset for Detection and Localization of Pedestrians and Vehicles](https://mlsanity.com/paper/2302.09429/) (2023)
- [A Novel Spike Transformer Network for Depth Estimation from Event Cameras via Cross-modality Knowledge Distillation](https://mlsanity.com/paper/2404.17335/) (2024)
- [Embedded event based object detection with spiking neural network](https://mlsanity.com/paper/2406.17617/) (2024)
- [A Time Encoding approach to training Spiking Neural Networks](https://mlsanity.com/paper/2110.06735/) (2021)
- [Parallel Hyperparameter Optimization Of Spiking Neural Network](https://mlsanity.com/paper/2403.00450/) (2024)
- [Spike-TBR: a Noise Resilient Neuromorphic Event Representation](https://mlsanity.com/paper/2506.04817/) (2025)
- [Benchmarking Microsaccade Recognition with Event Cameras: A Novel Dataset and Evaluation](https://mlsanity.com/paper/2510.24231/) (2025)
- [Topology Optimization of Random Memristors for Input-Aware Dynamic SNN](https://mlsanity.com/paper/2407.18625/) (2024)
- [Self-Supervised Event Representations: Towards Accurate, Real-Time Perception on SoC FPGAs](https://mlsanity.com/paper/2505.07556/) (2025)
- [Incorporating Visual Cortical Lateral Connection Properties into CNN: Recurrent Activation and Excitatory-Inhibitory Separation](https://mlsanity.com/paper/2509.15460/) (2025)

## Abstract (arXiv)

Convolutional neural networks (CNNs) are now the de facto solution for computer vision problems thanks to their impressive results and ease of learning. These networks are composed of layers of connected units called artificial neurons, loosely modeling the neurons in a biological brain. However, their implementation on conventional hardware (CPU/GPU) results in high power consumption, making their integration on embedded systems difficult. In a car for example, embedded algorithms have very high constraints in term of energy, latency and accuracy. To design more efficient computer vision algorithms, we propose to follow an end-to-end biologically inspired approach using event cameras and spiking neural networks (SNNs). Event cameras output asynchronous and sparse events, providing an incredibly efficient data source, but processing these events with synchronous and dense algorithms such as CNNs does not yield any significant benefits. To address this limitation, we use spiking neural networks (SNNs), which are more biologically realistic neural networks where units communicate using discrete spikes. Due to the nature of their operations, they are hardware friendly and energy-efficient, but training them still remains a challenge. Our method enables the training of sparse spiking convolutional neural networks directly on event data, using the popular deep learning framework PyTorch. The performances in terms of accuracy, sparsity and training time on the popular DVS128 Gesture Dataset make it possible to use this bio-inspired approach for the future embedding of real-time applications on low-power neuromorphic hardware.