mlsanityData to 14 Oct 2026

PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving

arXiv 2112.12610 ↗First posted 23 Dec 2021cs.CV cs.ROcited by 190

On the map: World models → World modeling from LiDAR generation

Abstract (arXiv)

The accelerating development of autonomous driving technology has placed greater demands on obtaining large amounts of high-quality data. Representative, labeled, real world data serves as the fuel for training deep learning networks, critical for improving self-driving perception algorithms. In this paper, we introduce PandaSet, the first dataset produced by a complete, high-precision autonomous vehicle sensor kit with a no-cost commercial license. The dataset was collected using one 360deg mechanical spinning LiDAR, one forward-facing, long-range LiDAR, and 6 cameras. The dataset contains more than 100 scenes, each of which is 8 seconds long, and provides 28 types of labels for object classification and 37 types of labels for semantic segmentation. We provide baselines for LiDAR-only 3D object detection, LiDAR-camera fusion 3D object detection and LiDAR point cloud segmentation. For more details about PandaSet and the development kit, see https://scale.com/open-datasets/pandaset.

Key facts. Cited by 190 papers in the map (data through 14 Oct 2026). Placed in World modeling from LiDAR generation, one of 11 directions in World models. Builds on 17 of its 26 references that are in the map. First posted on arXiv 23 Dec 2021.

Builds on

17 of 26 references.

References · 17

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  6. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  7. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  8. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  9. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  10. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  11. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  12. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
5 more
  1. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  2. 2019Fast Point R-CNNcited by 119
  3. 2019A*3D Dataset: Towards Autonomous Driving in Challenging Environmentscited by 65
  4. 2020Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detectioncited by 18
  5. 2020Cirrus: A Long-range Bi-pattern LiDAR Datasetcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 13; 2023: 30; 2024: 42; 2025: 49; 2026: 56.

190 papers.

Sort:

By area:

  1. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  2. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  3. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  4. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 297
  5. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  6. 2024OmniRe: Omni Urban Scene Reconstructioncited by 161
  7. 2023NeuRAD: Neural Rendering for Autonomous Drivingcited by 150
  8. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 141
  9. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  10. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  11. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  12. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Drivingcited by 90
38 more
  1. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restorationcited by 89
  2. 2023Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous drivingcited by 83
  3. 2023UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generationcited by 78
  4. 2022Multi-modal Sensor Fusion for Auto Driving Perception: A Surveycited by 76
  5. 2023Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveyscited by 75
  6. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 75
  7. 2024SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Drivingcited by 69
  8. 20253D and 4D World Modeling: A Surveycited by 67
  9. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  10. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 62
  11. 2022Uncertainty-aware Vision-based Metric Cross-view Geolocalizationcited by 54
  12. 2024AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstructioncited by 52
  13. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  14. 2023Three Pillars improving Vision Foundation Model Distillation for Lidarcited by 44
  15. 2026Driving on Registerscited by 44
  16. 20243DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Viewscited by 39
  17. 2023Real-Time Neural Rasterization for Large Scenescited by 37
  18. 2025ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representationcited by 36
  19. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  20. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 33
  21. 2024MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertaintycited by 32
  22. 2024G3R: Gradient Guided Generalizable Reconstructioncited by 31
  23. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  24. 2022CL3D: Unsupervised Domain Adaptation for Cross-LiDAR 3D Detectioncited by 27
  25. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  26. 2023CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulationcited by 26
  27. 2023Towards Unsupervised Object Detection From LiDAR Point Cloudscited by 23
  28. 2023Open-sourced Data Ecosystem in Autonomous Driving: the Present and Futurecited by 23
  29. 2023Reconstructing Objects in-the-wild for Realistic Sensor Simulationcited by 21
  30. 2024SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillationcited by 21
  31. 2023Deep Learning for Camera Calibration and Beyond: A Surveycited by 20
  32. 2023LightSim: Neural Lighting Simulation for Urban Scenescited by 20
  33. 2024Are NeRFs ready for autonomous driving? Towards closing the real-to-simulation gapcited by 20
  34. 2024Multiagent Multitraversal Multimodal Self-Driving: Open MARS Datasetcited by 18
  35. 2025R3D2: Realistic 3D Asset Insertion via Diffusion for Autonomous Driving Simulationcited by 18
  36. 2025Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorercited by 16
  37. 2024CMRNext: Camera to LiDAR Matching in the Wild for Localization and Extrinsic Calibrationcited by 15
  38. 2025Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splattingcited by 15

50 most cited of 190 All 190 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021LiDAR Cluster First and Camera Inference Later: A New Perspective Towards Autonomous Driving
  2. 2026TruckDrive: Long-Range Autonomous Highway Driving Dataset
  3. 2026The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset
  4. 2025CARScenes: Semantic VLM Dataset for Safe Autonomous Driving
  5. 2025RS2AD: End-to-End Autonomous Driving Data Generation from Roadside Sensor Observations
  6. 2026HilDA: Hierarchical Distillation with Diffusion for Advancing Self-Supervised LiDAR Pre-training
  7. 2026RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations
  8. 2025VESPA: Towards un(Human)supervised Open-World Pointcloud Labeling for Autonomous Driving