# Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks

arXiv: https://arxiv.org/abs/2302.08890 · HTML: https://mlsanity.com/paper/2302.08890/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- Builds on 141 of its 291 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Builds on (most cited first, 30 of 141)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Training Deep Spiking Neural Networks using Backpropagation](https://mlsanity.com/paper/1608.08782/) (2016)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [Deblur-NeRF: Neural Radiance Fields from Blurry Images](https://mlsanity.com/paper/2111.14292/) (2021)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [Video to Events: Recycling Video Datasets for Event Cameras](https://mlsanity.com/paper/1912.03095/) (2019)
- [Recurrent Vision Transformers for Object Detection with Event Cameras](https://mlsanity.com/paper/2212.05598/) (2022)

## Built on by (most cited first, 30 of 90)

- [UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All](https://mlsanity.com/paper/2403.12532/) (2024)
- [Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approach](https://mlsanity.com/paper/2404.00834/) (2024)
- [MLLMs are Deeply Affected by Modality Bias](https://mlsanity.com/paper/2505.18657/) (2025)
- [OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2405.05259/) (2024)
- [EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding](https://mlsanity.com/paper/2308.03135/) (2023)
- [Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution](https://mlsanity.com/paper/2303.13767/) (2023)
- [ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and More](https://mlsanity.com/paper/2403.12534/) (2024)
- [TMA: Temporal Motion Aggregation for Event-based Optical Flow](https://mlsanity.com/paper/2303.11629/) (2023)
- [Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalities](https://mlsanity.com/paper/2407.11351/) (2024)
- [EventGPT: Event Stream Understanding with Multimodal Large Language Models](https://mlsanity.com/paper/2412.00832/) (2024)
- [eTraM: Event-based Traffic Monitoring Dataset](https://mlsanity.com/paper/2403.19976/) (2024)
- [EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition](https://mlsanity.com/paper/2403.14082/) (2024)
- [Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation](https://mlsanity.com/paper/2407.11344/) (2024)
- [Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera](https://mlsanity.com/paper/2309.09297/) (2023)
- [Event-based Stereo Depth Estimation: A Survey](https://mlsanity.com/paper/2409.17680/) (2024)
- [Benchmarking Multi-modal Semantic Segmentation under Sensor Failures: Missing and Noisy Modality Robustness](https://mlsanity.com/paper/2503.18445/) (2025)
- [Event-based Simultaneous Localization and Mapping: A Comprehensive Survey](https://mlsanity.com/paper/2304.09793/) (2023)
- [EvLight++: Low-Light Video Enhancement with an Event Camera: A Large-Scale Real-World Dataset, Novel Method, and More](https://mlsanity.com/paper/2408.16254/) (2024)
- [SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Events](https://mlsanity.com/paper/2309.12842/) (2023)
- [Towards Omnidirectional Reasoning with 360-R1: A Dataset, Benchmark, and GRPO-based Method](https://mlsanity.com/paper/2505.14197/) (2025)
- [TIM: An Efficient Temporal Interaction Module for Spiking Transformer](https://mlsanity.com/paper/2401.11687/) (2024)
- [MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection](https://mlsanity.com/paper/2412.16876/) (2024)
- [PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Era](https://mlsanity.com/paper/2509.12989/) (2025)
- [eMoE-Tracker: Environmental MoE-based Transformer for Robust Event-guided Object Tracking](https://mlsanity.com/paper/2406.20024/) (2024)
- [Elite-EvGS: Learning Event-based 3D Gaussian Splatting by Distilling Event-to-Video Priors](https://mlsanity.com/paper/2409.13392/) (2024)
- [MemorySAM: Memorize Modalities and Semantics with Segment Anything Model 2 for Multi-modal Semantic Segmentation](https://mlsanity.com/paper/2503.06700/) (2025)
- [Unsupervised Joint Learning of Optical Flow and Intensity with Event Cameras](https://mlsanity.com/paper/2503.17262/) (2025)
- [EV-MGRFlowNet: Motion-Guided Recurrent Network for Unsupervised Event-based Optical Flow with Hybrid Motion-Compensation Loss](https://mlsanity.com/paper/2305.07853/) (2023)
- [UniINR: Event-guided Unified Rolling Shutter Correction, Deblurring, and Interpolation](https://mlsanity.com/paper/2305.15078/) (2023)
- [Image Anything: Towards Reasoning-coherent and Training-free Multi-modal Image Generation](https://mlsanity.com/paper/2401.17664/) (2024)

## Abstract (arXiv)

Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously and produce event streams encoding the time, pixel position, and polarity (sign) of the intensity changes. Event cameras possess a myriad of advantages over canonical frame-based cameras, such as high temporal resolution, high dynamic range, low latency, etc. Being capable of capturing information in challenging visual conditions, event cameras have the potential to overcome the limitations of frame-based cameras in the computer vision and robotics community. In very recent years, deep learning (DL) has been brought to this emerging field and inspired active research endeavors in mining its potential. However, there is still a lack of taxonomies in DL techniques for event-based vision. We first scrutinize the typical event representations with quality enhancement methods as they play a pivotal role as inputs to the DL models. We then provide a comprehensive survey of existing DL-based methods by structurally grouping them into two major categories: 1) image/video reconstruction and restoration; 2) event-based scene understanding and 3D vision. We conduct benchmark experiments for the existing methods in some representative research directions, i.e., image reconstruction, deblurring, and object recognition, to identify some critical insights and problems. Finally, we have discussions regarding the challenges and provide new perspectives for inspiring more research studies.