mlsanityData to 7 Oct 2026

Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting

arXiv 2301.00493 ↗First posted 2 Jan 2023cs.CV cs.AI cs.LGcited by 701

Not on the map. Why?

Cited by 104 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Argoverse 2 (AV2) - a collection of three datasets for perception and forecasting research in the self-driving domain. The annotated Sensor Dataset contains 1,000 sequences of multimodal data, encompassing high-resolution imagery from seven ring cameras, and two stereo cameras in addition to lidar point clouds, and 6-DOF map-aligned pose. Sequences contain 3D cuboid annotations for 26 object categories, all of which are sufficiently-sampled to support training and evaluation of 3D perception models. The Lidar Dataset contains 20,000 sequences of unlabeled lidar point clouds and map-aligned pose. This dataset is the largest ever collection of lidar sensor data and supports self-supervised learning and the emerging task of point cloud forecasting. Finally, the Motion Forecasting Dataset contains 250,000 scenarios mined for interesting and challenging interactions between the autonomous vehicle and other actors in each local scene. Models are tasked with the prediction of future motion for "scored actors" in each scenario and are provided with track histories that capture object location, heading, velocity, and category. In all three datasets, each scenario contains its own HD Map with 3D lane and crosswalk geometry - sourced from data captured in six distinct cities. We believe these datasets will support new and existing machine learning research problems in ways that existing datasets do not. All datasets are released under the CC BY-NC-SA 4.0 license.

Key facts. Cited by 701 papers in the map (data through 7 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 2 Jan 2023.

Builds on

42 of 58 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  7. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  8. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  9. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  10. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  11. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  12. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 280
30 more
  1. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  2. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  3. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  4. 2020A2D2: Audi Autonomous Driving Datasetcited by 189
  5. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  6. 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137
  7. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  8. 2021RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detectioncited by 115
  9. 2018TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agentscited by 111
  10. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  11. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  12. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  13. 2020Canadian Adverse Driving Conditions Datasetcited by 86
  14. 2019The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenescited by 80
  15. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  16. 2020Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detectioncited by 78
  17. 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
  18. 2020SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Drivingcited by 68
  19. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  20. 2019A*3D Dataset: Towards Autonomous Driving in Challenging Environmentscited by 64
  21. 2020AFDet: Anchor Free One Stage 3D Object Detectioncited by 63
  22. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  23. 2019Social Attention for Autonomous Decision-Making in Dense Trafficcited by 39
  24. 2020Cityscapes 3D: Dataset and Benchmark for 9 DoF Vehicle Detectioncited by 39
  25. 2021To the Point: Efficient 3D Object Detection in the Range Image with Graph Convolution Kernelscited by 39
  26. 2022Trust, but Verify: Cross-Modality Fusion for HD Map Change Detectioncited by 25
  27. 2021Lane Graph Estimation for Scene Understanding in Urban Drivingcited by 18
  28. 2020Streaming Object Detection for 3-D Point Cloudscited by 17
  29. 2022BANet: Motion Forecasting with Boundary Aware Networkcited by 10
  30. 2022QML for Argoverse 2 Motion Forecasting Challengecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 123; 2024: 198; 2025: 191; 2026: 189.

701 papers.

Sort:

By area:

  1. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  4. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  5. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  6. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  7. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  8. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  9. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  10. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  11. 2023VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Trackingcited by 221
  12. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
38 more
  1. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  2. 2024OmniRe: Omni Urban Scene Reconstructioncited by 159
  3. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  4. 2023NeuRAD: Neural Rendering for Autonomous Drivingcited by 146
  5. 2023StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Constructioncited by 143
  6. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  7. 2023V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecastingcited by 114
  8. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  9. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
  10. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  11. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  12. 2023The Waymo Open Sim Agents Challengecited by 86
  13. 2023PivotNet: Vectorized Pivot Learning for End-to-end HD Map Constructioncited by 84
  14. 2023OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mappingcited by 83
  15. 2023Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous drivingcited by 83
  16. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecastingcited by 76
  17. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
  18. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  19. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  20. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
  21. 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
  22. 2023Cross Modal Transformer: Towards Fast and Robust 3D Object Detectioncited by 68
  23. 2024MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mappingcited by 67
  24. 2024TUMTraf V2X Cooperative Perception Datasetcited by 66
  25. 2024SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Drivingcited by 66
  26. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  27. 20253D and 4D World Modeling: A Surveycited by 64
  28. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 61
  29. 2025A Survey of World Models for Autonomous Drivingcited by 60
  30. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  31. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  32. 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarioscited by 60
  33. 2023Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Mapscited by 58
  34. 2023LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Drivingcited by 57
  35. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  36. 2023Online Map Vectorization for Autonomous Driving: A Rasterization Perspectivecited by 54
  37. 2024P-MapNet: Far-seeing Map Generator Enhanced by both SDMap and HDMap Priorscited by 54
  38. 2023Graph-based Topology Reasoning for Driving Scenescited by 53

50 most cited of 701 All 701 (JSON)