# Optical Flow Estimation for Spiking Camera

arXiv: https://arxiv.org/abs/2110.03916 · HTML: https://mlsanity.com/paper/2110.03916/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 17 of 17)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2003.13045/) (2020)
- [Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness](https://mlsanity.com/paper/1608.05842/) (2016)
- [Accurate Optical Flow via Direct Cost Volume Processing](https://mlsanity.com/paper/1704.07325/) (2017)
- [Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation](https://mlsanity.com/paper/2109.04871/) (2021)
- [Deep Video Super-Resolution using HR Optical Flow Estimation](https://mlsanity.com/paper/2001.02129/) (2020)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)
- [An Efficient Coding Method for Spike Camera using Inter-Spike Intervals](https://mlsanity.com/paper/1912.09669/) (2019)

## Built on by (most cited first, 29 of 29)

- [Lightweight Event-based Optical Flow Estimation via Iterative Deblurring](https://mlsanity.com/paper/2211.13726/) (2022)
- [Towards Anytime Optical Flow Estimation with Event Cameras](https://mlsanity.com/paper/2307.05033/) (2023)
- [Temporal Event Stereo via Joint Learning with Stereoscopic Flow](https://mlsanity.com/paper/2407.10831/) (2024)
- [E-Motion: Future Motion Simulation via Event Sequence Diffusion](https://mlsanity.com/paper/2410.08649/) (2024)
- [Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer](https://mlsanity.com/paper/2208.12527/) (2022)
- [A Dynamic Graph CNN with Cross-Representation Distillation for Event-Based Recognition](https://mlsanity.com/paper/2302.04177/) (2023)
- [SpikeCV: Open a Continuous Computer Vision Era](https://mlsanity.com/paper/2303.11684/) (2023)
- [SpikeNeRF: Learning Neural Radiance Fields from Continuous Spike Stream](https://mlsanity.com/paper/2403.11222/) (2024)
- [Spike-NeRF: Neural Radiance Field Based On Spike Camera](https://mlsanity.com/paper/2403.16410/) (2024)
- [Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views](https://mlsanity.com/paper/2307.00821/) (2023)
- [SpikeGS: Reconstruct 3D scene via fast-moving bio-inspired sensors](https://mlsanity.com/paper/2407.03771/) (2024)
- [SOTA: Spike-Navigated Optimal TrAnsport Saliency Region Detection in Composite-bias Videos](https://mlsanity.com/paper/2505.00394/) (2025)
- [Finding Visual Saliency in Continuous Spike Stream](https://mlsanity.com/paper/2403.06233/) (2024)
- [Uncertainty Guided Depth Fusion for Spike Camera](https://mlsanity.com/paper/2208.12653/) (2022)
- [Highly Efficient SNNs for High-speed Object Detection](https://mlsanity.com/paper/2309.15883/) (2023)
- [SCSim: A Realistic Spike Cameras Simulator](https://mlsanity.com/paper/2405.16790/) (2024)
- [SpikeMM: Flexi-Magnification of High-Speed Micro-Motions](https://mlsanity.com/paper/2406.00383/) (2024)
- [SpikeGS: Learning 3D Gaussian Fields from Continuous Spike Stream](https://mlsanity.com/paper/2409.15176/) (2024)
- [Cricket: A Self-Powered Chirping Pixel](https://mlsanity.com/paper/2505.02246/) (2025)
- [SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams](https://mlsanity.com/paper/2505.19487/) (2025)
- [Spike Stream Denoising via Spike Camera Simulation](https://mlsanity.com/paper/2304.03129/) (2023)
- [Learning to Robustly Reconstruct Low-light Dynamic Scenes from Spike Streams](https://mlsanity.com/paper/2401.10461/) (2024)
- [Spike Imaging Velocimetry: Dense Motion Estimation of Fluids Using Spike Cameras](https://mlsanity.com/paper/2504.18864/) (2025)
- [SPACT18: Spiking Human Action Recognition Benchmark Dataset with Complementary RGB and Thermal Modalities](https://mlsanity.com/paper/2507.16151/) (2025)
- [Seeing the Unseen in Low-light Spike Streams](https://mlsanity.com/paper/2509.23304/) (2025)
- [SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streams](https://mlsanity.com/paper/2510.10602/) (2025)
- [Driving in Spikes: An Entropy-Guided Object Detector for Spike Cameras](https://mlsanity.com/paper/2511.15459/) (2025)
- [Optical Flow from Photons](https://mlsanity.com/paper/2608.00499/) (2026)
- [E-WAVE: Event-based Continuous Optical Flow via Warping-Aligned Visual Encoding](https://mlsanity.com/paper/2609.34346/) (2026)

## Abstract (arXiv)

As a bio-inspired sensor with high temporal resolution, the spiking camera has an enormous potential in real applications, especially for motion estimation in high-speed scenes. However, frame-based and event-based methods are not well suited to spike streams from the spiking camera due to the different data modalities. To this end, we present, SCFlow, a tailored deep learning pipeline to estimate optical flow in high-speed scenes from spike streams. Importantly, a novel input representation is introduced which can adaptively remove the motion blur in spike streams according to the prior motion. Further, for training SCFlow, we synthesize two sets of optical flow data for the spiking camera, SPIkingly Flying Things and Photo-realistic High-speed Motion, denoted as SPIFT and PHM respectively, corresponding to random high-speed and well-designed scenes. Experimental results show that the SCFlow can predict optical flow from spike streams in different high-speed scenes. Moreover, SCFlow shows promising generalization on real spike streams. Codes and datasets refer to https://github.com/Acnext/Optical-Flow-For-Spiking-Camera.