# SpikeMS: Deep Spiking Neural Network for Motion Segmentation

arXiv: https://arxiv.org/abs/2105.06562 · HTML: https://mlsanity.com/paper/2105.06562/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 17 of 17)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Training Deep Spiking Neural Networks using Backpropagation](https://mlsanity.com/paper/1608.08782/) (2016)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [Event-Based Motion Segmentation by Motion Compensation](https://mlsanity.com/paper/1904.01293/) (2019)
- [HFirst: A Temporal Approach to Object Recognition](https://mlsanity.com/paper/1508.01176/) (2015)
- [Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception](https://mlsanity.com/paper/1807.10936/) (2018)
- [EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras](https://mlsanity.com/paper/1903.07520/) (2019)
- [EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras](https://mlsanity.com/paper/1906.02919/) (2019)
- [Event-Based Angular Velocity Regression with Spiking Networks](https://mlsanity.com/paper/2003.02790/) (2020)
- [Real-time clustering and multi-target tracking using event-based sensors](https://mlsanity.com/paper/1807.02851/) (2018)
- [Spiking Optical Flow for Event-based Sensors Using IBM's TrueNorth Neurosynaptic System](https://mlsanity.com/paper/1710.09820/) (2017)
- [Event-based attention and tracking on neuromorphic hardware](https://mlsanity.com/paper/1907.04060/) (2019)
- [A Deep 2-Dimensional Dynamical Spiking Neuronal Network for Temporal Encoding trained with STDP](https://mlsanity.com/paper/2009.00581/) (2020)

## Built on by (most cited first, 29 of 29)

- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [Direct Learning-Based Deep Spiking Neural Networks: A Review](https://mlsanity.com/paper/2305.19725/) (2023)
- [StereoSpike: Depth Learning with a Spiking Neural Network](https://mlsanity.com/paper/2109.13751/) (2021)
- [Emergent Visual Sensors for Autonomous Vehicles](https://mlsanity.com/paper/2205.09383/) (2022)
- [Un-EVIMO: Unsupervised Event-Based Independent Motion Segmentation](https://mlsanity.com/paper/2312.00114/) (2023)
- [Out of the Room: Generalizing Event-Based Dynamic Motion Segmentation for Complex Scenes](https://mlsanity.com/paper/2403.04562/) (2024)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [JSTR: Joint Spatio-Temporal Reasoning for Event-based Moving Object Detection](https://mlsanity.com/paper/2403.07436/) (2024)
- [Instance-Level Moving Object Segmentation from a Single Image with Events](https://mlsanity.com/paper/2502.12975/) (2025)
- [2D versus 3D Convolutional Spiking Neural Networks Trained with Unsupervised STDP for Human Action Recognition](https://mlsanity.com/paper/2205.13474/) (2022)
- [Wandering around: A bioinspired approach to visual attention through object motion sensitivity](https://mlsanity.com/paper/2502.06747/) (2025)
- [Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow](https://mlsanity.com/paper/2503.06992/) (2025)
- [Neuromorphic Vision-based Motion Segmentation with Graph Transformer Neural Network](https://mlsanity.com/paper/2404.10940/) (2024)
- [EventSleep: Sleep Activity Recognition with Event Cameras](https://mlsanity.com/paper/2404.01801/) (2024)
- [Hardware, Algorithms, and Applications of the Neuromorphic Vision Sensor: a Review](https://mlsanity.com/paper/2504.08588/) (2025)
- [Spike-EVPR: Deep Spiking Residual Networks with SNN-Tailored Representations for Event-Based Visual Place Recognition](https://mlsanity.com/paper/2402.10476/) (2024)
- [Motion Segmentation for Neuromorphic Aerial Surveillance](https://mlsanity.com/paper/2405.15209/) (2024)
- [Retina-Inspired Object Motion Segmentation for Event-Cameras](https://mlsanity.com/paper/2408.09454/) (2024)
- [Iterative Event-based Motion Segmentation by Variational Contrast Maximization](https://mlsanity.com/paper/2504.18447/) (2025)
- [EvDetMAV: Generalized MAV Detection from Moving Event Cameras](https://mlsanity.com/paper/2506.19416/) (2025)
- [SPARTA: Advancing Sparse Attention in Spiking Neural Networks via Spike-Timing-Based Prioritization](https://mlsanity.com/paper/2508.01646/) (2025)
- [Spiking Two-Stream Methods with Unsupervised STDP-based Learning for Action Recognition](https://mlsanity.com/paper/2306.13783/) (2023)
- [GNEP Based Dynamic Segmentation and Motion Estimation for Neuromorphic Imaging](https://mlsanity.com/paper/2307.02595/) (2023)
- [Multiscale fusion enhanced spiking neural network for invasive BCI neural signal decoding](https://mlsanity.com/paper/2410.03533/) (2024)
- [EV-LayerSegNet: Self-supervised Motion Segmentation using Event Cameras](https://mlsanity.com/paper/2506.06596/) (2025)
- [Neuromorphic visual attention for Sign-language recognition on SpiNNaker](https://mlsanity.com/paper/2605.06005/) (2026)
- [Neuromorphic Object Detection: An In-Depth Study and Future Directions](https://mlsanity.com/paper/2607.23576/) (2026)
- [Can Spiking Neural Networks play pinball? A neuromorphic motion detector for target tracking](https://mlsanity.com/paper/2609.24403/) (2026)

## Abstract (arXiv)

Spiking Neural Networks (SNN) are the so-called third generation of neural networks which attempt to more closely match the functioning of the biological brain. They inherently encode temporal data, allowing for training with less energy usage and can be extremely energy efficient when coded on neuromorphic hardware. In addition, they are well suited for tasks involving event-based sensors, which match the event-based nature of the SNN. However, SNNs have not been as effectively applied to real-world, large-scale tasks as standard Artificial Neural Networks (ANNs) due to the algorithmic and training complexity. To exacerbate the situation further, the input representation is unconventional and requires careful analysis and deep understanding. In this paper, we propose textitSpikeMS, the first deep encoder-decoder SNN architecture for the real-world large-scale problem of motion segmentation using the event-based DVS camera as input. To accomplish this, we introduce a novel spatio-temporal loss formulation that includes both spike counts and classification labels in conjunction with the use of new techniques for SNN backpropagation. In addition, we show that textitSpikeMS is capable of textitincremental predictions, or predictions from smaller amounts of test data than it is trained on. This is invaluable for providing outputs even with partial input data for low-latency applications and those requiring fast predictions. We evaluated textitSpikeMS on challenging synthetic and real-world sequences from EV-IMO, EED and MOD datasets and achieving results on a par with a comparable ANN method, but using potentially 50 times less power.