# SKoPe3D: A Synthetic Dataset for Vehicle Keypoint Perception in 3D from Traffic Monitoring Cameras

arXiv: https://arxiv.org/abs/2309.01324 · HTML: https://mlsanity.com/paper/2309.01324/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving](https://mlsanity.com/paper/2005.03778/) (2020)
- [CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification](https://mlsanity.com/paper/1903.09254/) (2019)
- [Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task](https://mlsanity.com/paper/2203.13608/) (2022)
- [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019)
- [ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving](https://mlsanity.com/paper/1811.12222/) (2018)
- [Category-Specific Object Reconstruction from a Single Image](https://mlsanity.com/paper/1411.6069/) (2014)
- [Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor](https://mlsanity.com/paper/1504.02340/) (2015)
- [Cityscapes 3D: Dataset and Benchmark for 9 DoF Vehicle Detection](https://mlsanity.com/paper/2006.07864/) (2020)
- [BoxCars: Improving Fine-Grained Recognition of Vehicles using 3-D Bounding Boxes in Traffic Surveillance](https://mlsanity.com/paper/1703.00686/) (2017)
- [Sim-to-Real Domain Adaptation for Lane Detection and Classification in Autonomous Driving](https://mlsanity.com/paper/2202.07133/) (2022)
- [Synthehicle: Multi-Vehicle Multi-Camera Tracking in Virtual Cities](https://mlsanity.com/paper/2208.14167/) (2022)
- [Learning Optimal Parameters for Multi-target Tracking with Contextual Interactions](https://mlsanity.com/paper/1610.01394/) (2016)
- [CAROM Air -- Vehicle Localization and Traffic Scene Reconstruction from Aerial Videos](https://mlsanity.com/paper/2306.00075/) (2023)
- [CAROM -- Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures](https://mlsanity.com/paper/2104.00893/) (2021)
- [WorldGen: A Large Scale Generative Simulator](https://mlsanity.com/paper/2210.00715/) (2022)

## Built on by (most cited first, 1 of 1)

- [ToosiCubix: Monocular 3D Cuboid Labeling via Vehicle Part Annotations](https://mlsanity.com/paper/2506.21358/) (2025)

## Abstract (arXiv)

Intelligent transportation systems (ITS) have revolutionized modern road infrastructure, providing essential functionalities such as traffic monitoring, road safety assessment, congestion reduction, and law enforcement. Effective vehicle detection and accurate vehicle pose estimation are crucial for ITS, particularly using monocular cameras installed on the road infrastructure. One fundamental challenge in vision-based vehicle monitoring is keypoint detection, which involves identifying and localizing specific points on vehicles (such as headlights, wheels, taillights, etc.). However, this task is complicated by vehicle model and shape variations, occlusion, weather, and lighting conditions. Furthermore, existing traffic perception datasets for keypoint detection predominantly focus on frontal views from ego vehicle-mounted sensors, limiting their usability in traffic monitoring. To address these issues, we propose SKoPe3D, a unique synthetic vehicle keypoint dataset generated using the CARLA simulator from a roadside perspective. This comprehensive dataset includes generated images with bounding boxes, tracking IDs, and 33 keypoints for each vehicle. Spanning over 25k images across 28 scenes, SKoPe3D contains over 150k vehicle instances and 4.9 million keypoints. To demonstrate its utility, we trained a keypoint R-CNN model on our dataset as a baseline and conducted a thorough evaluation. Our experiments highlight the dataset's applicability and the potential for knowledge transfer between synthetic and real-world data. By leveraging the SKoPe3D dataset, researchers and practitioners can overcome the limitations of existing datasets, enabling advancements in vehicle keypoint detection for ITS.