# The Interstate-24 3D Dataset: a new benchmark for 3D multi-camera vehicle tracking

arXiv: https://arxiv.org/abs/2308.14833 · HTML: https://mlsanity.com/paper/2308.14833/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 59 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 23 of 23)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving](https://mlsanity.com/paper/2001.03343/) (2020)
- [CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification](https://mlsanity.com/paper/1903.09254/) (2019)
- [UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking](https://mlsanity.com/paper/1511.04136/) (2015)
- [Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular image](https://mlsanity.com/paper/1703.07570/) (2017)
- [Monocular 3D Object Detection: An Extrinsic Parameter Free Approach](https://mlsanity.com/paper/2106.15796/) (2021)
- [Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments](https://mlsanity.com/paper/1705.01693/) (2017)
- [Deep Multi-camera People Detection](https://mlsanity.com/paper/1702.04593/) (2017)
- [I-24 MOTION: An instrument for freeway traffic science](https://mlsanity.com/paper/2301.11198/) (2023)
- [BoxCars: Improving Fine-Grained Recognition of Vehicles using 3-D Bounding Boxes in Traffic Surveillance](https://mlsanity.com/paper/1703.00686/) (2017)
- [Monocular 3D Vehicle Detection Using Uncalibrated Traffic Cameras through Homography](https://mlsanity.com/paper/2103.15293/) (2021)
- [Are commercially implemented adaptive cruise control systems string stable?](https://mlsanity.com/paper/1905.02108/) (2019)
- [Synthehicle: Multi-Vehicle Multi-Camera Tracking in Virtual Cities](https://mlsanity.com/paper/2208.14167/) (2022)
- [Online Clustering-based Multi-Camera Vehicle Tracking in Scenarios with overlapping FOVs](https://mlsanity.com/paper/2102.04091/) (2021)
- [Automatic vehicle trajectory data reconstruction at scale](https://mlsanity.com/paper/2212.07907/) (2022)
- [Conditional Vehicle Trajectories Prediction in CARLA Urban Environment](https://mlsanity.com/paper/1909.00792/) (2019)
- [Multi-camera Multi-Object Tracking](https://mlsanity.com/paper/1709.07065/) (2017)

## Built on by (most cited first, 3 of 3)

- [Lifting Multi-View Detection and Tracking to the Bird's Eye View](https://mlsanity.com/paper/2403.12573/) (2024)
- [So you think you can track?](https://mlsanity.com/paper/2309.07268/) (2023)
- [Reinforcement Learning Based Oscillation Dampening: Scaling up Single-Agent RL algorithms to a 100 AV highway field operational test](https://mlsanity.com/paper/2402.17050/) (2024)

## Abstract (arXiv)

This work presents a novel video dataset recorded from overlapping highway traffic cameras along an urban interstate, enabling multi-camera 3D object tracking in a traffic monitoring context. Data is released from 3 scenes containing video from at least 16 cameras each, totaling 57 minutes in length. 877,000 3D bounding boxes and corresponding object tracklets are fully and accurately annotated for each camera field of view and are combined into a spatially and temporally continuous set of vehicle trajectories for each scene. Lastly, existing algorithms are combined to benchmark a number of 3D multi-camera tracking pipelines on the dataset, with results indicating that the dataset is challenging due to the difficulty of matching objects traveling at high speeds across cameras and heavy object occlusion, potentially for hundreds of frames, during congested traffic. This work aims to enable the development of accurate and automatic vehicle trajectory extraction algorithms, which will play a vital role in understanding impacts of autonomous vehicle technologies on the safety and efficiency of traffic.