# Cityscapes 3D: Dataset and Benchmark for 9 DoF Vehicle Detection

arXiv: https://arxiv.org/abs/2006.07864 · HTML: https://mlsanity.com/paper/2006.07864/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2020.

## Built on by (most cited first, 30 of 39)

- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild](https://mlsanity.com/paper/2207.10660/) (2022)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task](https://mlsanity.com/paper/2203.13608/) (2022)
- [3D Object Detection from Images for Autonomous Driving: A Survey](https://mlsanity.com/paper/2202.02980/) (2022)
- [Multi-Modal 3D Object Detection in Autonomous Driving: a Survey](https://mlsanity.com/paper/2106.12735/) (2021)
- [Detect Anything 3D in the Wild](https://mlsanity.com/paper/2504.07958/) (2025)
- [Data-driven Traffic Simulation: A Comprehensive Review](https://mlsanity.com/paper/2310.15975/) (2023)
- [Towards Robust Robot 3D Perception in Urban Environments: The UT Campus Object Dataset](https://mlsanity.com/paper/2309.13549/) (2023)
- [LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection](https://mlsanity.com/paper/2206.07705/) (2022)
- [Open Vocabulary Monocular 3D Object Detection](https://mlsanity.com/paper/2411.16833/) (2024)
- [InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding](https://mlsanity.com/paper/2306.04842/) (2023)
- [Ad-datasets: a meta-collection of data sets for autonomous driving](https://mlsanity.com/paper/2202.01909/) (2022)
- [IDD-3D: Indian Driving Dataset for 3D Unstructured Road Scenes](https://mlsanity.com/paper/2210.12878/) (2022)
- [Multi-task Learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generator](https://mlsanity.com/paper/2403.03468/) (2024)
- [Improving Distant 3D Object Detection Using 2D Box Supervision](https://mlsanity.com/paper/2403.09230/) (2024)
- [Every Dataset Counts: Scaling up Monocular 3D Object Detection with Joint Datasets Training](https://mlsanity.com/paper/2310.00920/) (2023)
- [A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects](https://mlsanity.com/paper/2506.19769/) (2025)
- [TruckDrive: Long-Range Autonomous Highway Driving Dataset](https://mlsanity.com/paper/2603.02413/) (2026)
- [Self-Supervised Object Detection via Generative Image Synthesis](https://mlsanity.com/paper/2110.09848/) (2021)
- [CAR -- Cityscapes Attributes Recognition A Multi-category Attributes Dataset for Autonomous Vehicles](https://mlsanity.com/paper/2111.08243/) (2021)
- [TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments](https://mlsanity.com/paper/2208.07943/) (2022)
- [Long Range Object-Level Monocular Depth Estimation for UAVs](https://mlsanity.com/paper/2302.08943/) (2023)
- [A Dynamic Feature Interaction Framework for Multi-task Visual Perception](https://mlsanity.com/paper/2306.05061/) (2023)
- [Bangladeshi Native Vehicle Detection in Wild](https://mlsanity.com/paper/2405.12150/) (2024)
- [Multi-Class Multi-Instance Count Conditioned Adversarial Image Generation](https://mlsanity.com/paper/2103.16795/) (2021)
- [Learning Cascaded Detection Tasks with Weakly-Supervised Domain Adaptation](https://mlsanity.com/paper/2107.04523/) (2021)

## Abstract (arXiv)

Detecting vehicles and representing their position and orientation in the three dimensional space is a key technology for autonomous driving. Recently, methods for 3D vehicle detection solely based on monocular RGB images gained popularity. In order to facilitate this task as well as to compare and drive state-of-the-art methods, several new datasets and benchmarks have been published. Ground truth annotations of vehicles are usually obtained using lidar point clouds, which often induces errors due to imperfect calibration or synchronization between both sensors. To this end, we propose Cityscapes 3D, extending the original Cityscapes dataset with 3D bounding box annotations for all types of vehicles. In contrast to existing datasets, our 3D annotations were labeled using stereo RGB images only and capture all nine degrees of freedom. This leads to a pixel-accurate reprojection in the RGB image and a higher range of annotations compared to lidar-based approaches. In order to ease multitask learning, we provide a pairing of 2D instance segments with 3D bounding boxes. In addition, we complement the Cityscapes benchmark suite with 3D vehicle detection based on the new annotations as well as metrics presented in this work. Dataset and benchmark are available online.