# Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception

arXiv: https://arxiv.org/abs/1807.10936 · HTML: https://mlsanity.com/paper/1807.10936/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2018.

## Built on by (most cited first, 30 of 48)

- [Event-Based Fusion for Motion Deblurring with Cross-modal Attention](https://mlsanity.com/paper/2112.00167/) (2021)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Learning Dense and Continuous Optical Flow from an Event Camera](https://mlsanity.com/paper/2211.09078/) (2022)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation](https://mlsanity.com/paper/2109.04871/) (2021)
- [Taming Contrast Maximization for Learning Sequential, Low-latency, Event-based Optical Flow](https://mlsanity.com/paper/2303.05214/) (2023)
- [Event Collapse in Contrast Maximization Frameworks](https://mlsanity.com/paper/2207.04007/) (2022)
- [Fully neuromorphic vision and control for autonomous drone flight](https://mlsanity.com/paper/2303.08778/) (2023)
- [SpikeMS: Deep Spiking Neural Network for Motion Segmentation](https://mlsanity.com/paper/2105.06562/) (2021)
- [Real-Time Optical Flow for Vehicular Perception with Low- and High-Resolution Event Cameras](https://mlsanity.com/paper/2112.10591/) (2021)
- [ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks](https://mlsanity.com/paper/2110.12211/) (2021)
- [SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition](https://mlsanity.com/paper/2308.04369/) (2023)
- [Emergent Visual Sensors for Autonomous Vehicles](https://mlsanity.com/paper/2205.09383/) (2022)
- [Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics](https://mlsanity.com/paper/2209.11741/) (2022)
- [Microsaccade-inspired Event Camera for Robotics](https://mlsanity.com/paper/2405.17769/) (2024)
- [Event-based Tiny Object Detection: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2506.23575/) (2025)
- [Temporal Event Stereo via Joint Learning with Stereoscopic Flow](https://mlsanity.com/paper/2407.10831/) (2024)
- [Fast Event-based Optical Flow Estimation by Triplet Matching](https://mlsanity.com/paper/2212.12218/) (2022)
- [Globally Optimal Event-Based Divergence Estimation for Ventral Landing](https://mlsanity.com/paper/2209.13168/) (2022)
- [EV-MGRFlowNet: Motion-Guided Recurrent Network for Unsupervised Event-based Optical Flow with Hybrid Motion-Compensation Loss](https://mlsanity.com/paper/2305.07853/) (2023)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks](https://mlsanity.com/paper/2503.04223/) (2025)
- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [An optimised deep spiking neural network architecture without gradients](https://mlsanity.com/paper/2109.12813/) (2021)
- [Vector-Symbolic Architecture for Event-Based Optical Flow](https://mlsanity.com/paper/2405.08300/) (2024)
- [Hardware, Algorithms, and Applications of the Neuromorphic Vision Sensor: a Review](https://mlsanity.com/paper/2504.08588/) (2025)
- [6-DoF Object Tracking with Event-based Optical Flow and Frames](https://mlsanity.com/paper/2508.14776/) (2025)
- [Voltage-Dependent Synaptic Plasticity (VDSP): Unsupervised probabilistic Hebbian plasticity rule based on neurons membrane potential](https://mlsanity.com/paper/2203.11022/) (2022)
- [Fusing Frame and Event Vision for High-speed Optical Flow for Edge Application](https://mlsanity.com/paper/2207.10720/) (2022)

## Abstract (arXiv)

The combination of spiking neural networks and event-based vision sensors holds the potential of highly efficient and high-bandwidth optical flow estimation. This paper presents the first hierarchical spiking architecture in which motion (direction and speed) selectivity emerges in an unsupervised fashion from the raw stimuli generated with an event-based camera. A novel adaptive neuron model and stable spike-timing-dependent plasticity formulation are at the core of this neural network governing its spike-based processing and learning, respectively. After convergence, the neural architecture exhibits the main properties of biological visual motion systems, namely feature extraction and local and global motion perception. Convolutional layers with input synapses characterized by single and multiple transmission delays are employed for feature and local motion perception, respectively; while global motion selectivity emerges in a final fully-connected layer. The proposed solution is validated using synthetic and real event sequences. Along with this paper, we provide the cuSNN library, a framework that enables GPU-accelerated simulations of large-scale spiking neural networks. Source code and samples are available at https://github.com/tudelft/cuSNN.