# IPS300+: a Challenging Multimodal Dataset for Intersection Perception System

arXiv: https://arxiv.org/abs/2106.02781 · HTML: https://mlsanity.com/paper/2106.02781/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [Disentangling Monocular 3D Object Detection](https://mlsanity.com/paper/1905.12365/) (2019)
- [SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation](https://mlsanity.com/paper/2002.10111/) (2020)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving](https://mlsanity.com/paper/2001.03343/) (2020)
- [Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud](https://mlsanity.com/paper/1903.09847/) (2019)
- [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019)
- [End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection](https://mlsanity.com/paper/2004.03080/) (2020)
- [Center3D: Center-based Monocular 3D Object Detection with Joint Depth Understanding](https://mlsanity.com/paper/2005.13423/) (2020)
- [Monocular 3D Detection with Geometric Constraints Embedding and Semi-supervised Training](https://mlsanity.com/paper/2009.00764/) (2020)

## Built on by (most cited first, 26 of 26)

- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [InfraDet3D: Multi-Modal 3D Object Detection based on Roadside Infrastructure Camera and LiDAR Sensors](https://mlsanity.com/paper/2305.00314/) (2023)
- [PointSSC: A Cooperative Vehicle-Infrastructure Point Cloud Benchmark for Semantic Scene Completion](https://mlsanity.com/paper/2309.12708/) (2023)
- [Collaborative Perception Datasets in Autonomous Driving: A Survey](https://mlsanity.com/paper/2404.14022/) (2024)
- [A Survey of Robust 3D Object Detection Methods in Point Clouds](https://mlsanity.com/paper/2204.00106/) (2022)
- [Real-Time And Robust 3D Object Detection with Roadside LiDARs](https://mlsanity.com/paper/2207.05200/) (2022)
- [R-LiViT: A LiDAR-Visual-Thermal Dataset Enabling Vulnerable Road User Focused Roadside Perception](https://mlsanity.com/paper/2503.17122/) (2025)
- [Collaborative Perception Datasets for Autonomous Driving: A Review](https://mlsanity.com/paper/2504.12696/) (2025)
- [An Efficient Semi-Automated Scheme for Infrastructure LiDAR Annotation](https://mlsanity.com/paper/2301.10732/) (2023)
- [PointSee: Image Enhances Point Cloud](https://mlsanity.com/paper/2211.01664/) (2022)
- [Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2405.19202/) (2024)
- [A9 Intersection Dataset: All You Need for Urban 3D Camera-LiDAR Roadside Perception](https://mlsanity.com/paper/2306.09266/) (2023)
- [Collective Perception Datasets for Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2405.16973/) (2024)
- [FedRSU: Federated Learning for Scene Flow Estimation on Roadside Units](https://mlsanity.com/paper/2401.12862/) (2024)
- [Towards Vision Zero: The TUM Traffic Accid3nD Dataset](https://mlsanity.com/paper/2503.12095/) (2025)
- [UrbanTwin: Synthetic Roadside LiDAR Datasets](https://mlsanity.com/paper/2509.06781/) (2025)
- [All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles](https://mlsanity.com/paper/2510.26641/) (2025)
- [CORP: A Multi-Modal Dataset for Campus-Oriented Roadside Perception Tasks](https://mlsanity.com/paper/2404.03191/) (2024)
- [Cooperative Visual-LiDAR Extrinsic Calibration Technology for Intersection Vehicle-Infrastructure: A review](https://mlsanity.com/paper/2405.10132/) (2024)
- [Enhancing Highway Safety: Accident Detection on the A9 Test Stretch Using Roadside Sensors](https://mlsanity.com/paper/2502.00402/) (2025)
- [Safety-Critical Learning for Long-Tail Events: The TUM Traffic Accident Dataset](https://mlsanity.com/paper/2508.14567/) (2025)
- [RESOLVE: A Multi-Resolution and Multi-Modal Dataset for Roadside Cooperative Perception](https://mlsanity.com/paper/2606.31895/) (2026)
- [Framework for Quality Evaluation of Smart Roadside Infrastructure Sensors for Automated Driving Applications](https://mlsanity.com/paper/2304.07745/) (2023)
- [LiGuard: A Streamlined Open-Source Framework for Rapid & Interactive Lidar Research](https://mlsanity.com/paper/2509.02902/) (2025)
- [UrbanTwin: Building High-Fidelity Digital Twins for Sim2Real LiDAR Perception and Evaluation](https://mlsanity.com/paper/2509.02903/) (2025)
- [High-Fidelity Digital Twins for Bridging the Sim2Real Gap in LiDAR-Based ITS Perception](https://mlsanity.com/paper/2509.02904/) (2025)

## Abstract (arXiv)

Due to the high complexity and occlusion, insufficient perception in the crowded urban intersection can be a serious safety risk for both human drivers and autonomous algorithms, whereas CVIS (Cooperative Vehicle Infrastructure System) is a proposed solution for full-participants perception in this scenario. However, the research on roadside multimodal perception is still in its infancy, and there is no open-source dataset for such scenario. Accordingly, this paper fills the gap. Through an IPS (Intersection Perception System) installed at the diagonal of the intersection, this paper proposes a high-quality multimodal dataset for the intersection perception task. The center of the experimental intersection covers an area of 3000m2, and the extended distance reaches 300m, which is typical for CVIS. The first batch of open-source data includes 14198 frames, and each frame has an average of 319.84 labels, which is 9.6 times larger than the most crowded dataset (H3D dataset in 2019) by now. In order to facilitate further study, this dataset tries to keep the label documents consistent with the KITTI dataset, and a standardized benchmark is created for algorithm evaluation. Our dataset is available at: http://www.openmpd.com/column/other_datasets.