mlsanityData to 14 Oct 2026

V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting

arXiv 2305.05938 ↗First posted 10 May 2023cs.CV cs.AIcited by 114

Not on the map. Why?

Abstract (arXiv)

Utilizing infrastructure and vehicle-side information to track and forecast the behaviors of surrounding traffic participants can significantly improve decision-making and safety in autonomous driving. However, the lack of real-world sequential datasets limits research in this area. To address this issue, we introduce V2X-Seq, the first large-scale sequential V2X dataset, which includes data frames, trajectories, vector maps, and traffic lights captured from natural scenery. V2X-Seq comprises two parts: the sequential perception dataset, which includes more than 15,000 frames captured from 95 scenarios, and the trajectory forecasting dataset, which contains about 80,000 infrastructure-view scenarios, 80,000 vehicle-view scenarios, and 50,000 cooperative-view scenarios captured from 28 intersections' areas, covering 672 hours of data. Based on V2X-Seq, we introduce three new tasks for vehicle-infrastructure cooperative (VIC) autonomous driving: VIC3D Tracking, Online-VIC Forecasting, and Offline-VIC Forecasting. We also provide benchmarks for the introduced tasks. Find data, code, and more up-to-date information at hrefhttps://github.com/AIR-THU/DAIR-V2X-Seqhttps://github.com/AIR-THU/DAIR-V2X-Seq.

Key facts. Cited by 114 papers in the map (data through 14 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

36 of 44 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  5. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  6. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  7. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  8. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  9. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  10. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  11. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  12. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
24 more
  1. 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 277
  2. 2022DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 273
  3. 2022Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Mapscited by 255
  4. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 240
  5. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  6. 2021Learning Distilled Collaboration Graph for Multi-Agent Perceptioncited by 194
  7. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 190
  8. 2019Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Cloudscited by 165
  9. 2020When2com: Multi-Agent Perception via Communication Graph Groupingcited by 157
  10. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  11. 2022V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Drivingcited by 157
  12. 2020Who2com: Collaborative Perception via Learnable Handshake Communicationcited by 127
  13. 2018TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agentscited by 111
  14. 2019Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensorscited by 110
  15. 2022COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehiclescited by 95
  16. 2022Latency-Aware Collaborative Perceptioncited by 86
  17. 2022Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Taskcited by 81
  18. 2020AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metricscited by 63
  19. 2022A9-Dataset: Multi-Sensor Infrastructure-Based Dataset for Mobility Researchcited by 40
  20. 2021Score refinement for confidence-based 3D multi-object trackingcited by 32
  21. 2021AutoCast: Scalable Infrastructure-less Cooperative Perception for Distributed Collaborative Drivingcited by 26
  22. 2023Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Predictioncited by 25
  23. 2019Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networkscited by 15
  24. 2021Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Datacited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 35; 2025: 42; 2026: 24.

114 papers.

Sort:

By area:

  1. 2024Communication-Efficient Collaborative Perception via Information Filling with Codebookcited by 68
  2. 2024TUMTraf V2X Cooperative Perception Datasetcited by 67
  3. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  4. 2024ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictionscited by 66
  5. 2023FRNet: Frustum-Range Networks for Scalable LiDAR Segmentationcited by 50
  6. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  7. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  8. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  9. 2024RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perceptioncited by 38
  10. 2023DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detectioncited by 34
  11. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  12. 2024MEDCO: Medical Education Copilots Based on A Multi-Agent Frameworkcited by 33
38 more
  1. 2023Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 31
  2. 2024V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Predictioncited by 31
  3. 2023BEVHeight++: Toward Robust Visual Centric 3D Object Detectioncited by 30
  4. 2025CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridizationcited by 28
  5. 2023QUEST: Query Stream for Practical Cooperative Perceptioncited by 23
  6. 2023MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddingscited by 22
  7. 2024V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perceptioncited by 22
  8. 2023Learning Cooperative Trajectory Representations for Motion Forecastingcited by 21
  9. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  10. 2023MonoLSS: Learnable Sample Selection For Monocular 3D Detectioncited by 19
  11. 2023Probabilistic 3D Multi-Object Cooperative Tracking for Autonomous Driving via Differentiable Multi-Sensor Kalman Filtercited by 15
  12. 2023Vehicle-to-Everything Cooperative Perception for Autonomous Drivingcited by 15
  13. 2024BEVSpread: Spread Voxel Pooling for Bird's-Eye-View Representation in Vision-based Roadside 3D Object Detectioncited by 15
  14. 2025Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmarkcited by 15
  15. 2025SparseAlign: A Fully Sparse Framework for Cooperative Object Detectioncited by 15
  16. 2025TraF-Align: Trajectory-aware Feature Alignment for Asynchronous Multi-agent Perceptioncited by 15
  17. 2025UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perceptioncited by 15
  18. 2023PointSSC: A Cooperative Vehicle-Infrastructure Point Cloud Benchmark for Semantic Scene Completioncited by 14
  19. 2024HoloVIC: Large-scale Dataset and Benchmark for Multi-Sensor Holographic Intersection and Vehicle-Infrastructure Cooperativecited by 14
  20. 2025Co-MTP: A Cooperative Trajectory Prediction Framework with Multi-Temporal Fusion for Autonomous Drivingcited by 14
  21. 2025V2X-LLM: Enhancing V2X Integration and Understanding in Connected Vehicle Corridorscited by 13
  22. 2025CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perceptioncited by 12
  23. 2024Collaborative Perception Datasets in Autonomous Driving: A Surveycited by 11
  24. 2024CooPre: Cooperative Pretraining for V2X Cooperative Perceptioncited by 11
  25. 2025CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splattingcited by 11
  26. 2024V2X-PC: Vehicle-to-everything Collaborative Perception via Point Clustercited by 10
  27. 2024RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perceptioncited by 9
  28. 2025Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competitioncited by 9
  29. 2025TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Predictioncited by 9
  30. 2024Robust Collaborative Perception without External Localization and Clock Devicescited by 8
  31. 2025V2X-ReaLO: An Open Online Framework and Dataset for Cooperative Perception in Realitycited by 8
  32. 2025Collaborative Perception Datasets for Autonomous Driving: A Reviewcited by 8
  33. 2025Highly Accurate and Diverse Traffic Data: The DeepScenario Open 3D Datasetcited by 7
  34. 2024Breaking Data Silos: Cross-Domain Learning for Multi-Agent Perception from Independent Private Sourcescited by 5
  35. 2024RopeBEV: A Multi-Camera Roadside Perception Network in Bird's-Eye-Viewcited by 5
  36. 2025Mixed Signals: A Diverse Point Cloud Dataset for Heterogeneous LiDAR V2X Collaborationcited by 5
  37. 2025Systematic Literature Review on Vehicular Collaborative Perception -- A Computer Vision Perspectivecited by 5
  38. 2025M3CAD: Towards Generic Cooperative Autonomous Driving Benchmarkcited by 5

50 most cited of 114 All 114 (JSON)