# Secrets of Event-Based Optical Flow

arXiv: https://arxiv.org/abs/2207.10022 · HTML: https://mlsanity.com/paper/2207.10022/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 60 references that are in the map.
- First posted on arXiv 20 Jul 2022.

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [Focus Is All You Need: Loss Functions For Event-based Vision](https://mlsanity.com/paper/1904.07235/) (2019)
- [Event-Based Motion Segmentation by Motion Compensation](https://mlsanity.com/paper/1904.01293/) (2019)
- [Event-based Motion Segmentation with Spatio-Temporal Graph Cuts](https://mlsanity.com/paper/2012.08730/) (2020)
- [Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation](https://mlsanity.com/paper/2109.04871/) (2021)
- [Event Collapse in Contrast Maximization Frameworks](https://mlsanity.com/paper/2207.04007/) (2022)
- [Globally-Optimal Contrast Maximisation for Event Cameras](https://mlsanity.com/paper/2206.05127/) (2022)
- [The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data](https://mlsanity.com/paper/2106.06887/) (2021)
- [Real-Time Optical Flow for Vehicular Perception with Low- and High-Resolution Event Cameras](https://mlsanity.com/paper/2112.10591/) (2021)
- [Real-time high speed motion prediction using fast aperture-robust event-driven visual flow](https://mlsanity.com/paper/1811.11135/) (2018)
- [Simultaneous Optical Flow and Segmentation (SOFAS) using Dynamic Vision Sensor](https://mlsanity.com/paper/1805.12326/) (2018)

## Built on by (most cited first, 30 of 92)

- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Taming Contrast Maximization for Learning Sequential, Low-latency, Event-based Optical Flow](https://mlsanity.com/paper/2303.05214/) (2023)
- [Optical flow estimation from event-based cameras and spiking neural networks](https://mlsanity.com/paper/2302.06492/) (2023)
- [BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation](https://mlsanity.com/paper/2303.07716/) (2023)
- [TMA: Temporal Motion Aggregation for Event-based Optical Flow](https://mlsanity.com/paper/2303.11629/) (2023)
- [Lightweight Event-based Optical Flow Estimation via Iterative Deblurring](https://mlsanity.com/paper/2211.13726/) (2022)
- [CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization](https://mlsanity.com/paper/2403.08119/) (2024)
- [Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events](https://mlsanity.com/paper/2308.09383/) (2023)
- [A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild](https://mlsanity.com/paper/2301.05191/) (2023)
- [Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation](https://mlsanity.com/paper/2407.10802/) (2024)
- [Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion](https://mlsanity.com/paper/2207.10494/) (2022)
- [Event Camera Data Dense Pre-training](https://mlsanity.com/paper/2311.11533/) (2023)
- [Event-based Stereo Depth Estimation: A Survey](https://mlsanity.com/paper/2409.17680/) (2024)
- [ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras](https://mlsanity.com/paper/2410.09374/) (2024)
- [A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework](https://mlsanity.com/paper/2212.07350/) (2022)
- [Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion](https://mlsanity.com/paper/2311.00436/) (2023)
- [EventHDR: from Event to High-Speed HDR Videos and Beyond](https://mlsanity.com/paper/2409.17029/) (2024)
- [Towards Anytime Optical Flow Estimation with Event Cameras](https://mlsanity.com/paper/2307.05033/) (2023)
- [Event-based Continuous Color Video Decompression from Single Frames](https://mlsanity.com/paper/2312.00113/) (2023)
- [Temporal Event Stereo via Joint Learning with Stereoscopic Flow](https://mlsanity.com/paper/2407.10831/) (2024)
- [Motion and Structure from Event-based Normal Flow](https://mlsanity.com/paper/2407.12239/) (2024)
- [E-Motion: Future Motion Simulation via Event Sequence Diffusion](https://mlsanity.com/paper/2410.08649/) (2024)
- [Fast Event-based Optical Flow Estimation by Triplet Matching](https://mlsanity.com/paper/2212.12218/) (2022)
- [Event-based Background-Oriented Schlieren](https://mlsanity.com/paper/2311.00434/) (2023)
- [Unsupervised Joint Learning of Optical Flow and Intensity with Event Cameras](https://mlsanity.com/paper/2503.17262/) (2025)
- [EV-MGRFlowNet: Motion-Guided Recurrent Network for Unsupervised Event-based Optical Flow with Hybrid Motion-Compensation Loss](https://mlsanity.com/paper/2305.07853/) (2023)
- [Event-Free Moving Object Segmentation from Moving Ego Vehicle](https://mlsanity.com/paper/2305.00126/) (2023)
- [Un-EVIMO: Unsupervised Event-Based Independent Motion Segmentation](https://mlsanity.com/paper/2312.00114/) (2023)
- [EvTexture: Event-driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2406.13457/) (2024)

## Abstract (arXiv)

Event cameras respond to scene dynamics and offer advantages to estimate motion. Following recent image-based deep-learning achievements, optical flow estimation methods for event cameras have rushed to combine those image-based methods with event data. However, it requires several adaptations (data conversion, loss function, etc.) as they have very different properties. We develop a principled method to extend the Contrast Maximization framework to estimate optical flow from events alone. We investigate key elements: how to design the objective function to prevent overfitting, how to warp events to deal better with occlusions, and how to improve convergence with multi-scale raw events. With these key elements, our method ranks first among unsupervised methods on the MVSEC benchmark, and is competitive on the DSEC benchmark. Moreover, our method allows us to expose the issues of the ground truth flow in those benchmarks, and produces remarkable results when it is transferred to unsupervised learning settings. Our code is available at https://github.com/tub-rip/event_based_optical_flow