# DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Dataset

arXiv: https://arxiv.org/abs/2406.17908 · HTML: https://mlsanity.com/paper/2406.17908/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 25 of 25)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents](https://mlsanity.com/paper/1811.02146/) (2018)
- [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019)
- [A*3D Dataset: Towards Autonomous Driving in Challenging Environments](https://mlsanity.com/paper/1909.07541/) (2019)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff](https://mlsanity.com/paper/2102.09527/) (2021)
- [Real-Time Digital Twins: Vision and Research Directions for 6G and Beyond](https://mlsanity.com/paper/2301.11283/) (2023)
- [Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets](https://mlsanity.com/paper/2111.07574/) (2021)
- [Rethinking of Radar's Role: A Camera-Radar Dataset and Systematic Annotator via Coordinate Alignment](https://mlsanity.com/paper/2105.05207/) (2021)
- [LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications](https://mlsanity.com/paper/2203.05548/) (2022)
- [Radar Aided 6G Beam Prediction: Deep Learning Algorithms and Real-World Demonstration](https://mlsanity.com/paper/2111.09676/) (2021)
- [Position Aided Beam Prediction in the Real World: How Useful GPS Locations Actually Are?](https://mlsanity.com/paper/2205.09054/) (2022)
- [Integrated Sensing and Communication for 6G: Ten Key Machine Learning Roles](https://mlsanity.com/paper/2208.02157/) (2022)
- [Multi-Modal Beam Prediction Challenge 2022: Towards Generalization](https://mlsanity.com/paper/2209.07519/) (2022)
- [Computer Vision Aided Blockage Prediction in Real-World Millimeter Wave Deployments](https://mlsanity.com/paper/2203.01907/) (2022)
- [Towards Real-World 6G Drone Communication: Position and Camera Aided Beam Prediction](https://mlsanity.com/paper/2205.12187/) (2022)
- [Proactively Predicting Dynamic 6G Link Blockages Using LiDAR and In-Band Signatures](https://mlsanity.com/paper/2211.09535/) (2022)

## Built on by (most cited first, 4 of 4)

- [ENWAR: A RAG-empowered Multi-Modal LLM Framework for Wireless Environment Perception](https://mlsanity.com/paper/2410.18104/) (2024)
- [Multi-Modality Sensing in mmWave Beamforming for Connected Vehicles Using Deep Learning](https://mlsanity.com/paper/2504.06173/) (2025)
- [Multi-Modal Sensing Aided mmWave Beamforming for V2V Communications with Transformers](https://mlsanity.com/paper/2509.11112/) (2025)
- [Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework](https://mlsanity.com/paper/2602.13606/) (2026)

## Abstract (arXiv)

High data rate and low-latency vehicle-to-vehicle (V2V) communication are essential for future intelligent transport systems to enable coordination, enhance safety, and support distributed computing and intelligence requirements. Developing effective communication strategies, however, demands realistic test scenarios and datasets. This is important at the high-frequency bands where more spectrum is available, yet harvesting this bandwidth is challenged by the need for direction transmission and the sensitivity of signal propagation to blockages. This work presents the first large-scale multi-modal dataset for studying mmWave vehicle-to-vehicle communications. It presents a two-vehicle testbed that comprises data from a 360-degree camera, four radars, four 60 GHz phased arrays, a 3D lidar, and two precise GPSs. The dataset contains vehicles driving during the day and night for 120 km in intercity and rural settings, with speeds up to 100 km per hour. More than one million objects were detected across all images, from trucks to bicycles. This work further includes detailed dataset statistics that prove the coverage of various situations and highlights how this dataset can enable novel machine-learning applications.