# Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving

arXiv: https://arxiv.org/abs/2305.02008 · HTML: https://mlsanity.com/paper/2305.02008/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 3 May 2023.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving](https://mlsanity.com/paper/2112.12610/) (2021)
- [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019)
- [The Mapillary Traffic Sign Dataset for Detection and Classification on a Global Scale](https://mlsanity.com/paper/1909.04422/) (2019)

## Built on by (most cited first, 30 of 83)

- [NeuRAD: Neural Rendering for Autonomous Driving](https://mlsanity.com/paper/2311.15260/) (2023)
- [SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving](https://mlsanity.com/paper/2411.16816/) (2024)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Driving](https://mlsanity.com/paper/2404.07762/) (2024)
- [Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix It](https://mlsanity.com/paper/2312.06420/) (2023)
- [MAN TruckScenes: A multimodal dataset for autonomous trucking in diverse conditions](https://mlsanity.com/paper/2407.07462/) (2024)
- [Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis](https://mlsanity.com/paper/2506.11526/) (2025)
- [MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertainty](https://mlsanity.com/paper/2401.12761/) (2024)
- [DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Model](https://mlsanity.com/paper/2410.10738/) (2024)
- [Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities](https://mlsanity.com/paper/2401.08045/) (2024)
- [DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving](https://mlsanity.com/paper/2601.01528/) (2026)
- [Are NeRFs ready for autonomous driving? Towards closing the real-to-simulation gap](https://mlsanity.com/paper/2403.16092/) (2024)
- [Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset](https://mlsanity.com/paper/2406.09383/) (2024)
- [R3D2: Realistic 3D Asset Insertion via Diffusion for Autonomous Driving Simulation](https://mlsanity.com/paper/2506.07826/) (2025)
- [Perspective-Invariant 3D Object Detection](https://mlsanity.com/paper/2507.17665/) (2025)
- [PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Driving](https://mlsanity.com/paper/2507.17596/) (2025)
- [MS3D++: Ensemble of Experts for Multi-Source Unsupervised Domain Adaptation in 3D Object Detection](https://mlsanity.com/paper/2308.05988/) (2023)
- [HiMo: High-Speed Objects Motion Compensation in Point Clouds](https://mlsanity.com/paper/2503.00803/) (2025)
- [DriveDiTFit: Fine-tuning Diffusion Transformers for Autonomous Driving](https://mlsanity.com/paper/2407.15661/) (2024)
- [Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2503.14537/) (2025)
- [R-LiViT: A LiDAR-Visual-Thermal Dataset Enabling Vulnerable Road User Focused Roadside Perception](https://mlsanity.com/paper/2503.17122/) (2025)
- [Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching](https://mlsanity.com/paper/2512.01850/) (2025)
- [DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method](https://mlsanity.com/paper/2508.17054/) (2025)
- [Bosch Street Dataset: A Multi-Modal Dataset with Imaging Radar for Automated Driving](https://mlsanity.com/paper/2407.12803/) (2024)
- [NeuRadar: Neural Radiance Fields for Automotive Radar Point Clouds](https://mlsanity.com/paper/2504.00859/) (2025)
- [World-Action Models for Robot Learning and Control: A Survey](https://mlsanity.com/paper/2609.16074/) (2026)
- [Towards Long-Range 3D Object Detection for Autonomous Vehicles](https://mlsanity.com/paper/2310.04800/) (2023)
- [Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2405.19202/) (2024)
- [XLM for Autonomous Driving Systems: A Comprehensive Review](https://mlsanity.com/paper/2409.10484/) (2024)
- [Hard Cases Detection in Motion Prediction by Vision-Language Foundation Models](https://mlsanity.com/paper/2405.20991/) (2024)

## Abstract (arXiv)

Existing datasets for autonomous driving (AD) often lack diversity and long-range capabilities, focusing instead on 360deg perception and temporal reasoning. To address this gap, we introduce Zenseact Open Dataset (ZOD), a large-scale and diverse multimodal dataset collected over two years in various European countries, covering an area 9x that of existing datasets. ZOD boasts the highest range and resolution sensors among comparable datasets, coupled with detailed keyframe annotations for 2D and 3D objects (up to 245m), road instance/semantic segmentation, traffic sign recognition, and road classification. We believe that this unique combination will facilitate breakthroughs in long-range perception and multi-task learning. The dataset is composed of Frames, Sequences, and Drives, designed to encompass both data diversity and support for spatio-temporal learning, sensor fusion, localization, and mapping. Frames consist of 100k curated camera images with two seconds of other supporting sensor data, while the 1473 Sequences and 29 Drives include the entire sensor suite for 20 seconds and a few minutes, respectively. ZOD is the only large-scale AD dataset released under a permissive license, allowing for both research and commercial use. More information, and an extensive devkit, can be found at https://zod.zenseact.com