# DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction

arXiv: https://arxiv.org/abs/2005.08605 · HTML: https://mlsanity.com/paper/2005.08605/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 30 of 56)

- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021)
- [TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset](https://mlsanity.com/paper/2108.07329/) (2021)
- [EVIMO2: An Event Camera Dataset for Motion Segmentation, Optical Flow, Structure from Motion, and Visual Inertial Odometry in Indoor Scenes with Monocular or Stereo Algorithms](https://mlsanity.com/paper/2205.03467/) (2022)
- [CarSNN: An Efficient Spiking Neural Network for Event-Based Autonomous Cars on the Loihi Neuromorphic Research Processor](https://mlsanity.com/paper/2107.00401/) (2021)
- [Ev-TTA: Test-Time Adaptation for Event-Based Object Recognition](https://mlsanity.com/paper/2203.12247/) (2022)
- [Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset](https://mlsanity.com/paper/2412.06647/) (2024)
- [Emergent Visual Sensors for Autonomous Vehicles](https://mlsanity.com/paper/2205.09383/) (2022)
- [Dataset and Benchmark: Novel Sensors for Autonomous Vehicle Perception](https://mlsanity.com/paper/2401.13853/) (2024)
- [SEVD: Synthetic Event-based Vision Dataset for Ego and Fixed Traffic Perception](https://mlsanity.com/paper/2404.10540/) (2024)
- [A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services](https://mlsanity.com/paper/2304.14271/) (2023)
- [Deep Visual Odometry with Events and Frames](https://mlsanity.com/paper/2309.09947/) (2023)
- [TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras](https://mlsanity.com/paper/2401.08474/) (2024)
- [Ad-datasets: a meta-collection of data sets for autonomous driving](https://mlsanity.com/paper/2202.01909/) (2022)
- [FE-Fusion-VPR: Attention-based Multi-Scale Network Architecture for Visual Place Recognition by Fusing Frames and Events](https://mlsanity.com/paper/2211.12244/) (2022)
- [Enhancing Visual Place Recognition via Fast and Slow Adaptive Biasing in Event Cameras](https://mlsanity.com/paper/2403.16425/) (2024)
- [ADV2E: Bridging the Gap Between Analogue Circuit and Discrete Frames in the Video-to-Events Simulator](https://mlsanity.com/paper/2411.12250/) (2024)
- [NYC-Event-VPR: A Large-Scale High-Resolution Event-Based Visual Place Recognition Dataset in Dense Urban Environments](https://mlsanity.com/paper/2410.21615/) (2024)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [CoSEC: A Coaxial Stereo Event Camera Dataset for Autonomous Driving](https://mlsanity.com/paper/2408.08500/) (2024)
- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [SuperEvent: Cross-Modal Learning of Event-based Keypoint Detection for SLAM](https://mlsanity.com/paper/2504.00139/) (2025)
- [An Asynchronous Intensity Representation for Framed and Event Video Sources](https://mlsanity.com/paper/2301.08783/) (2023)
- [Learning to Estimate Two Dense Depths from LiDAR and Event Data](https://mlsanity.com/paper/2302.14444/) (2023)
- [Ensemble-Based Event Camera Place Recognition Under Varying Illumination](https://mlsanity.com/paper/2509.01968/) (2025)
- [Multi-Modal Fusion for Sensorimotor Coordination in Steering Angle Prediction](https://mlsanity.com/paper/2202.05500/) (2022)

## Abstract (arXiv)

Neuromorphic event cameras are useful for dynamic vision problems under difficult lighting conditions. To enable studies of using event cameras in automobile driving applications, this paper reports a new end-to-end driving dataset called DDD20. The dataset was captured with a DAVIS camera that concurrently streams both dynamic vision sensor (DVS) brightness change events and active pixel sensor (APS) intensity frames. DDD20 is the longest event camera end-to-end driving dataset to date with 51h of DAVIS event+frame camera and vehicle human control data collected from 4000km of highway and urban driving under a variety of lighting conditions. Using DDD20, we report the first study of fusing brightness change events and intensity frame data using a deep learning approach to predict the instantaneous human steering wheel angle. Over all day and night conditions, the explained variance for human steering prediction from a Resnet-32 is significantly better from the fused DVS+APS frames (0.88) than using either DVS (0.67) or APS (0.77) data alone.