# Learning Dense and Continuous Optical Flow from an Event Camera

arXiv: https://arxiv.org/abs/2211.09078 · HTML: https://mlsanity.com/paper/2211.09078/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 72 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [Video to Events: Recycling Video Datasets for Event Cameras](https://mlsanity.com/paper/1912.03095/) (2019)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2003.13045/) (2020)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation](https://mlsanity.com/paper/1809.05571/) (2018)
- [Learning Event-Based Motion Deblurring](https://mlsanity.com/paper/2004.05794/) (2020)
- [SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warping](https://mlsanity.com/paper/2105.07014/) (2021)

## Built on by (most cited first, 30 of 60)

- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Event Camera Data Pre-training](https://mlsanity.com/paper/2301.01928/) (2023)
- [Taming Contrast Maximization for Learning Sequential, Low-latency, Event-based Optical Flow](https://mlsanity.com/paper/2303.05214/) (2023)
- [OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2405.05259/) (2024)
- [TMA: Temporal Motion Aggregation for Event-based Optical Flow](https://mlsanity.com/paper/2303.11629/) (2023)
- [Learning Optical Flow from Event Camera with Rendered Dataset](https://mlsanity.com/paper/2303.11011/) (2023)
- [RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2309.15082/) (2023)
- [Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation](https://mlsanity.com/paper/2407.10802/) (2024)
- [Event Camera Data Dense Pre-training](https://mlsanity.com/paper/2311.11533/) (2023)
- [Towards Anytime Optical Flow Estimation with Event Cameras](https://mlsanity.com/paper/2307.05033/) (2023)
- [BlinkVision: A Benchmark for Optical Flow, Scene Flow and Point Tracking Estimation using RGB Frames and Events](https://mlsanity.com/paper/2410.20451/) (2024)
- [Temporal Event Stereo via Joint Learning with Stereoscopic Flow](https://mlsanity.com/paper/2407.10831/) (2024)
- [E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning](https://mlsanity.com/paper/2311.18433/) (2023)
- [SoDaCam: Software-defined Cameras via Single-Photon Imaging](https://mlsanity.com/paper/2309.00066/) (2023)
- [EvTexture: Event-driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2406.13457/) (2024)
- [EMatch: A Unified Framework for Event-based Optical Flow and Stereo Matching](https://mlsanity.com/paper/2407.21735/) (2024)
- [Instance-Level Moving Object Segmentation from a Single Image with Events](https://mlsanity.com/paper/2502.12975/) (2025)
- [EDCFlow: Exploring Temporally Dense Difference Maps for Event-based Optical Flow Estimation](https://mlsanity.com/paper/2506.03512/) (2025)
- [Segment Any Events via Weighted Adaptation of Pivotal Tokens](https://mlsanity.com/paper/2312.16222/) (2023)
- [Learning Normal Flow Directly From Event Neighborhoods](https://mlsanity.com/paper/2412.11284/) (2024)
- [Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow](https://mlsanity.com/paper/2503.06992/) (2025)
- [Dark-EvGS: Event Camera as an Eye for Radiance Field in the Dark](https://mlsanity.com/paper/2507.11931/) (2025)
- [CrossZoom: Simultaneously Motion Deblurring and Event Super-Resolving](https://mlsanity.com/paper/2309.16949/) (2023)
- [Vector-Symbolic Architecture for Event-Based Optical Flow](https://mlsanity.com/paper/2405.08300/) (2024)
- [MATE: Motion-Augmented Temporal Consistency for Event-based Point Tracking](https://mlsanity.com/paper/2412.01300/) (2024)
- [Spatially-guided Temporal Aggregation for Robust Event-RGB Optical Flow Estimation](https://mlsanity.com/paper/2501.00838/) (2025)
- [V2V: Scaling Event-Based Vision through Efficient Video-to-Voxel Simulation](https://mlsanity.com/paper/2505.16797/) (2025)
- [Learning Efficient Meshflow and Optical Flow from Event Cameras](https://mlsanity.com/paper/2510.04111/) (2025)
- [Neuromorphic Synergy for Video Binarization](https://mlsanity.com/paper/2402.12644/) (2024)
- [ResFlow: Fine-tuning Residual Optical Flow for Event-based High Temporal Resolution Motion Estimation](https://mlsanity.com/paper/2412.09105/) (2024)

## Abstract (arXiv)

Event cameras such as DAVIS can simultaneously output high temporal resolution events and low frame-rate intensity images, which own great potential in capturing scene motion, such as optical flow estimation. Most of the existing optical flow estimation methods are based on two consecutive image frames and can only estimate discrete flow at a fixed time interval. Previous work has shown that continuous flow estimation can be achieved by changing the quantities or time intervals of events. However, they are difficult to estimate reliable dense flow , especially in the regions without any triggered events. In this paper, we propose a novel deep learning-based dense and continuous optical flow estimation framework from a single image with event streams, which facilitates the accurate perception of high-speed motion. Specifically, we first propose an event-image fusion and correlation module to effectively exploit the internal motion from two different modalities of data. Then we propose an iterative update network structure with bidirectional training for optical flow prediction. Therefore, our model can estimate reliable dense flow as two-frame-based methods, as well as estimate temporal continuous flow as event-based methods. Extensive experimental results on both synthetic and real captured datasets demonstrate that our model outperforms existing event-based state-of-the-art methods and our designed baselines for accurate dense and continuous optical flow estimation.