# DSEC: A Stereo Event Camera Dataset for Driving Scenarios

arXiv: https://arxiv.org/abs/2103.06011 · HTML: https://mlsanity.com/paper/2103.06011/

## Key facts

- Cited by 303 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 10 Mar 2021.

## Builds on (most cited first, 27 of 27)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [A Large Scale Event-based Detection Dataset for Automotive](https://mlsanity.com/paper/2001.08499/) (2020)
- [DDD17: End-To-End DAVIS Driving Dataset](https://mlsanity.com/paper/1711.01458/) (2017)
- [Learning Event-Based Motion Deblurring](https://mlsanity.com/paper/2004.05794/) (2020)
- [Semi-Dense 3D Reconstruction with a Stereo Event Camera](https://mlsanity.com/paper/1807.07429/) (2018)
- [DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction](https://mlsanity.com/paper/2005.08605/) (2020)
- [Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer](https://mlsanity.com/paper/1511.03240/) (2015)
- [Event-based visual place recognition with ensembles of temporal windows](https://mlsanity.com/paper/2006.02826/) (2020)
- [Tightly Coupled 3D Lidar Inertial Odometry and Mapping](https://mlsanity.com/paper/1904.06993/) (2019)
- [Single Image Optical Flow Estimation with an Event Camera](https://mlsanity.com/paper/2004.00347/) (2020)
- [Realtime Time Synchronized Event-based Stereo](https://mlsanity.com/paper/1803.09025/) (2018)
- [Asynchronous Spatial Image Convolutions for Event Cameras](https://mlsanity.com/paper/1812.00438/) (2018)

## Built on by (most cited first, 30 of 303)

- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Recurrent Vision Transformers for Object Detection with Event Cameras](https://mlsanity.com/paper/2212.05598/) (2022)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [Event-aided Direct Sparse Odometry](https://mlsanity.com/paper/2204.07640/) (2022)
- [Secrets of Event-Based Optical Flow](https://mlsanity.com/paper/2207.10022/) (2022)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [ESS: Learning Event-based Semantic Segmentation from Still Images](https://mlsanity.com/paper/2203.10016/) (2022)
- [State Space Models for Event Cameras](https://mlsanity.com/paper/2402.15584/) (2024)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Learning Dense and Continuous Optical Flow from an Event Camera](https://mlsanity.com/paper/2211.09078/) (2022)
- [How to Calibrate Your Event Camera](https://mlsanity.com/paper/2105.12362/) (2021)
- [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021)
- [Event Camera Data Pre-training](https://mlsanity.com/paper/2301.01928/) (2023)
- [Open Challenges in Deep Stereo: the Booster Dataset](https://mlsanity.com/paper/2206.04671/) (2022)
- [Hierarchical Neural Memory Network for Low Latency Event Processing](https://mlsanity.com/paper/2305.17852/) (2023)
- [Dense Continuous-Time Optical Flow from Events and Frames](https://mlsanity.com/paper/2203.13674/) (2022)
- [NeRF-Supervised Deep Stereo](https://mlsanity.com/paper/2303.17603/) (2023)
- [VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM](https://mlsanity.com/paper/2207.01404/) (2022)
- [TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset](https://mlsanity.com/paper/2108.07329/) (2021)
- [Masked Event Modeling: Self-Supervised Pretraining for Event Cameras](https://mlsanity.com/paper/2212.10368/) (2022)
- [RGB-Event Fusion for Moving Object Detection in Autonomous Driving](https://mlsanity.com/paper/2209.08323/) (2022)
- [Taming Contrast Maximization for Learning Sequential, Low-latency, Event-based Optical Flow](https://mlsanity.com/paper/2303.05214/) (2023)
- [OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2405.05259/) (2024)
- [ViViD++: Vision for Visibility Dataset](https://mlsanity.com/paper/2204.06183/) (2022)
- [Seeing Motion at Nighttime with an Event Camera](https://mlsanity.com/paper/2404.11884/) (2024)

## Abstract (arXiv)

Once an academic venture, autonomous driving has received unparalleled corporate funding in the last decade. Still, the operating conditions of current autonomous cars are mostly restricted to ideal scenarios. This means that driving in challenging illumination conditions such as night, sunrise, and sunset remains an open problem. In these cases, standard cameras are being pushed to their limits in terms of low light and high dynamic range performance. To address these challenges, we propose, DSEC, a new dataset that contains such demanding illumination conditions and provides a rich set of sensory data. DSEC offers data from a wide-baseline stereo setup of two color frame cameras and two high-resolution monochrome event cameras. In addition, we collect lidar data and RTK GPS measurements, both hardware synchronized with all camera data. One of the distinctive features of this dataset is the inclusion of high-resolution event cameras. Event cameras have received increasing attention for their high temporal resolution and high dynamic range performance. However, due to their novelty, event camera datasets in driving scenarios are rare. This work presents the first high-resolution, large-scale stereo dataset with event cameras. The dataset contains 53 sequences collected by driving in a variety of illumination conditions and provides ground truth disparity for the development and evaluation of event-based stereo algorithms.