mlsanityData to 13 Oct 2026

V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

arXiv 2202.08449 ↗First posted 17 Feb 2022cs.CVcited by 157

Not on the map. Why?

Abstract (arXiv)

Vehicle-to-everything (V2X) communication techniques enable the collaboration between vehicles and many other entities in the neighboring environment, which could fundamentally improve the perception system for autonomous driving. However, the lack of a public dataset significantly restricts the research progress of collaborative perception. To fill this gap, we present V2X-Sim, a comprehensive simulated multi-agent perception dataset for V2X-aided autonomous driving. V2X-Sim provides: (1) hlmulti-agent sensor recordings from the road-side unit (RSU) and multiple vehicles that enable collaborative perception, (2) multi-modality sensor streams that facilitate multi-modality perception, and (3) diverse ground truths that support various perception tasks. Meanwhile, we build an open-source testbed and provide a benchmark for the state-of-the-art collaborative perception algorithms on three tasks, including detection, tracking and segmentation. V2X-Sim seeks to stimulate collaborative perception research for autonomous driving before realistic datasets become widely available. Our dataset and code are available at urlhttps://ai4ce.github.io/V2X-Sim/.

Key facts. Cited by 157 papers in the map (data through 13 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 17 Feb 2022.

Builds on

32 of 52 references.

References · 32

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,656
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,498
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,435
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,440
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,439
  6. 2016Simple Online and Realtime Trackingcited by 864
  7. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 730
  8. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  9. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 549
  10. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 533
  11. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  12. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
20 more
  1. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  2. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 193
  3. 2019Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Cloudscited by 165
  4. 2020When2com: Multi-Agent Perception via Communication Graph Groupingcited by 157
  5. 2019WoodScape: A multi-task, multi-camera fisheye dataset for autonomous drivingcited by 144
  6. 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137
  7. 2020Who2com: Collaborative Perception via Learnable Handshake Communicationcited by 127
  8. 2019Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensorscited by 110
  9. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  10. 2020Canadian Adverse Driving Conditions Datasetcited by 86
  11. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 84
  12. 2019The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenescited by 80
  13. 2020Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challengescited by 80
  14. 2019A*3D Dataset: Towards Autonomous Driving in Challenging Environmentscited by 64
  15. 2019Deep Learning for Visual Tracking: A Comprehensive Surveycited by 60
  16. 2021Keypoints-Based Deep Feature Fusion for Cooperative Vehicle Detection of Autonomous Drivingcited by 45
  17. 2021Fast and Robust Registration of Partially Overlapping Point Cloudscited by 22
  18. 2021Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Datacited by 11
  19. 2020Securing Vehicle-to-Everything (V2X) Communication Platformscited by 9
  20. 2017VANETs Meet Autonomous Vehicles: A Multimodal 3D Environment Learning Approachcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 6; 2023: 31; 2024: 56; 2025: 44; 2026: 20.

157 papers.

Sort:

By area:

  1. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  2. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 308
  3. 2022Robust Collaborative 3D Object Detection in Presence of Pose Errorscited by 129
  4. 2023V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecastingcited by 114
  5. 2023Collaborative Perception in Autonomous Driving: Methods, Datasets and Challengescited by 87
  6. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  7. 2023Collaboration Helps Camera Overtake LiDAR in 3D Detectioncited by 81
  8. 2023BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detectioncited by 74
  9. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 74
  10. 2023DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Drivingcited by 73
  11. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  12. 2024TUMTraf V2X Cooperative Perception Datasetcited by 66
38 more
  1. 2023Asynchrony-Robust Collaborative Perception via Bird's Eye View Flowcited by 58
  2. 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
  3. 2022Uncertainty Quantification of Collaborative Detection for Self-Drivingcited by 46
  4. 2025STAMP: Scalable Task And Model-agnostic Collaborative Perceptioncited by 44
  5. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  6. 2024AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learningcited by 40
  7. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  8. 2023Among Us: Adversarially Robust Collaborative Perception by Consensuscited by 37
  9. 2024RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perceptioncited by 37
  10. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  11. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  12. 2024Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunitiescited by 32
  13. 2023Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 31
  14. 2024V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Predictioncited by 31
  15. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  16. 2023UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Frameworkcited by 29
  17. 2022V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perceptioncited by 28
  18. 2023Collaborative Multi-Object Tracking with Conformal Uncertainty Propagationcited by 27
  19. 2025V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Modelscited by 27
  20. 2025CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridizationcited by 27
  21. 2023Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Predictioncited by 25
  22. 2023Interruption-Aware Cooperative Perception for V2X Communication-Aided Autonomous Drivingcited by 25
  23. 2023Open-sourced Data Ecosystem in Autonomous Driving: the Present and Futurecited by 23
  24. 2022A Survey and Framework of Cooperative Perception: From Heterogeneous Singleton to Hierarchical Cooperationcited by 21
  25. 2024V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perceptioncited by 21
  26. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  27. 2023CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementaritycited by 18
  28. 2024Multiagent Multitraversal Multimodal Self-Driving: Open MARS Datasetcited by 18
  29. 2024V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detectioncited by 18
  30. 2023DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perceptioncited by 17
  31. 2024UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehiclescited by 16
  32. 2023Probabilistic 3D Multi-Object Cooperative Tracking for Autonomous Driving via Differentiable Multi-Sensor Kalman Filtercited by 15
  33. 2023Vehicle-to-Everything Cooperative Perception for Autonomous Drivingcited by 15
  34. 2025Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmarkcited by 15
  35. 2025UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perceptioncited by 15
  36. 2024HoloVIC: Large-scale Dataset and Benchmark for Multi-Sensor Holographic Intersection and Vehicle-Infrastructure Cooperativecited by 14
  37. 2024Directed-CP: Directed Collaborative Perception for Connected and Autonomous Vehicles via Proactive Attentioncited by 14
  38. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14

50 most cited of 157 All 157 (JSON)