# Neuromorphic Optical Flow and Real-time Implementation with Event Cameras

arXiv: https://arxiv.org/abs/2304.07139 · HTML: https://mlsanity.com/paper/2304.07139/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 14 Apr 2023.

## Builds on (most cited first, 22 of 22)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [Efficient Neuromorphic Signal Processing with Loihi 2](https://mlsanity.com/paper/2111.03746/) (2021)
- [FlowFusion: Dynamic Dense RGB-D SLAM Based on Optical Flow](https://mlsanity.com/paper/2003.05102/) (2020)
- [Real-Time Optical Flow for Vehicular Perception with Low- and High-Resolution Event Cameras](https://mlsanity.com/paper/2112.10591/) (2021)
- [Fast Event-based Optical Flow Estimation by Triplet Matching](https://mlsanity.com/paper/2212.12218/) (2022)
- [Mapping Spiking Neural Networks to Neuromorphic Hardware](https://mlsanity.com/paper/1909.01843/) (2019)
- [Kraken: A Direct Event/Frame-Based Multi-sensor Fusion SoC for Ultra-Efficient Visual Processing in Nano-UAVs](https://mlsanity.com/paper/2209.01065/) (2022)
- [Object Tracking in Satellite Videos Based on a Multi-Frame Optical Flow Tracker](https://mlsanity.com/paper/1804.09323/) (2018)

## Built on by (most cited first, 11 of 11)

- [Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware](https://mlsanity.com/paper/2312.00425/) (2023)
- [Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network](https://mlsanity.com/paper/2304.11857/) (2023)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [Event-based Optical Flow on Neuromorphic Processor: ANN vs. SNN Comparison based on Activation Sparsification](https://mlsanity.com/paper/2407.20421/) (2024)
- [PRE-Mamba: A 4D State Space Model for Ultra-High-Frequent Event Camera Deraining](https://mlsanity.com/paper/2505.05307/) (2025)
- [Vector-Symbolic Architecture for Event-Based Optical Flow](https://mlsanity.com/paper/2405.08300/) (2024)
- [Camera Motion Estimation from RGB-D-Inertial Scene Flow](https://mlsanity.com/paper/2404.17251/) (2024)
- [Spatio-Temporal State Space Model For Efficient Event-Based Optical Flow](https://mlsanity.com/paper/2506.07878/) (2025)
- [Event-driven Robust Fitting on Neuromorphic Hardware](https://mlsanity.com/paper/2508.09466/) (2025)
- [EventShiftFlow: Towards Hardware-efficient FPGA-based Flow Estimation](https://mlsanity.com/paper/2605.28312/) (2026)
- [Robust PnP on a Neuromorphic Processor for Object Pose Estimation](https://mlsanity.com/paper/2607.16834/) (2026)

## Abstract (arXiv)

Optical flow provides information on relative motion that is an important component in many computer vision pipelines. Neural networks provide high accuracy optical flow, yet their complexity is often prohibitive for application at the edge or in robots, where efficiency and latency play crucial role. To address this challenge, we build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture, inspired by Timelens, that improves the state-of-the-art self-supervised optical flow accuracy when operated both in spiking and non-spiking mode. To implement a real-time pipeline with a physical event camera, we propose a methodology for principled model simplification based on activity and latency analysis. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity while maintaining the accuracy, opening the path for real-time deployments.