mlsanityData to 7 Oct 2026

nuScenes: A multimodal dataset for autonomous driving

arXiv 1903.11027 ↗First posted 26 Mar 2019cs.LG cs.CV cs.ROcited by 4,485

Not on the map. Why?

Cited by 630 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We define novel 3D detection and tracking metrics. We also provide careful dataset analysis as well as baselines for lidar and image based detection and tracking. Data, development kit and more information are available online.

Key facts. Cited by 4,485 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 306; 2022: 496; 2023: 799; 2024: 983; 2025: 1064; 2026: 835.

4,485 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  3. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  4. 2022Planning-oriented Autonomous Drivingcited by 1,068
  5. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  6. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  7. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  8. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  9. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  10. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  11. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  12. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
38 more
  1. 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
  2. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  3. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  4. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  5. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  6. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  8. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  9. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  10. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  11. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  12. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  13. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  14. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  15. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  16. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  17. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  18. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  19. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  20. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  21. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
  22. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  23. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  24. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  25. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  26. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  27. 2022MAGVIT: Masked Generative Video Transformercited by 341
  28. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  29. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  30. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  31. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  32. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  33. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  34. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  35. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  36. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  37. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  38. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309

50 most cited of 4,485 All 4,485 (JSON)