mlsanityData to 7 Oct 2026

UniPAD: A Universal Pre-training Paradigm for Autonomous Driving

arXiv 2310.08370 ↗First posted 12 Oct 2023cs.CVcited by 73

Not on the map. Why?

Abstract (arXiv)

In the context of autonomous driving, the significance of effective feature learning is widely acknowledged. While conventional 3D self-supervised pre-training methods have shown widespread success, most methods follow the ideas originally designed for 2D images. In this paper, we present UniPAD, a novel self-supervised learning paradigm applying 3D volumetric differentiable rendering. UniPAD implicitly encodes 3D space, facilitating the reconstruction of continuous 3D shape structures and the intricate appearance characteristics of their 2D projections. The flexibility of our method enables seamless integration into both 2D and 3D frameworks, enabling a more holistic comprehension of the scenes. We manifest the feasibility and effectiveness of UniPAD by conducting extensive experiments on various downstream 3D tasks. Our method significantly improves lidar-, camera-, and lidar-camera-based baseline by 9.1, 7.7, and 6.9 NDS, respectively. Notably, our pre-training pipeline achieves 73.2 NDS for 3D object detection and 79.4 mIoU for 3D semantic segmentation on the nuScenes validation set, achieving state-of-the-art results in comparison with previous methods. The code will be available at https://github.com/Nightmare-n/UniPAD.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 97 of its 113 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

97 of 113 references.

References · 97

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2023Segment Anythingcited by 8,158
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2017Mask R-CNNcited by 7,259
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2022A ConvNet for the 2020scited by 3,545
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
38 more
  1. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  2. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  3. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  4. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  5. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  7. 2022Planning-oriented Autonomous Drivingcited by 1,068
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  9. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  10. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  11. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  13. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  14. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  15. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  16. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  17. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  18. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  19. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  20. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  21. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  22. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  23. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  24. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  25. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 409
  26. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  27. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
  28. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  29. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  30. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  31. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  32. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  33. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  34. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  35. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  36. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  37. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  38. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232

50 most cited of 97 All 97 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 24; 2025: 32; 2026: 12.

73 papers.

Sort:

By area:

  1. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  2. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  3. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Drivingcited by 89
  4. 2025A Survey of World Models for Autonomous Drivingcited by 60
  5. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  6. 2024DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Inputcited by 53
  7. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
  8. 2024Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learningcited by 45
  9. 2024OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentationcited by 43
  10. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecastingcited by 40
  11. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 36
  12. 2024DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Featurescited by 33
38 more
  1. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 33
  2. 2024SPA: 3D Spatial-Awareness Enables Effective Embodied Representationcited by 28
  3. 2024GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understandingcited by 26
  4. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  5. 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 22
  6. 2025EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesiscited by 20
  7. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19
  8. 2024OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networkscited by 17
  9. 2024GGS: Generalizable Gaussian Splatting for Lane Switching in Autonomous Drivingcited by 17
  10. 2024VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Drivingcited by 15
  11. 2023Applications of Large Scale Foundation Models for Autonomous Drivingcited by 14
  12. 2024AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Drivingcited by 14
  13. 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusioncited by 14
  14. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  15. 2024X-Drive: Cross-modality consistent multi-sensor data synthesis for driving scenarioscited by 13
  16. 2024GaussianPretrain: A Simple Unified 3D Gaussian Representation for Visual Pre-training in Autonomous Drivingcited by 11
  17. 2025Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Surveycited by 9
  18. 2025TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Predictioncited by 9
  19. 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Predictioncited by 9
  20. 2024MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learningcited by 8
  21. 2025Self-Supervised Sparse Sensor Fusion for Long Range Perceptioncited by 8
  22. 2024Self-supervised Learning of LiDAR 3D Point Clouds via 2D-3D Neural Calibrationcited by 7
  23. 2023Prospective Role of Foundation Models in Advancing Autonomous Vehiclescited by 7
  24. 2024Efficient Depth-Guided Urban View Synthesiscited by 7
  25. 2025Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Cloudscited by 7
  26. 2025NeuRadar: Neural Radiance Fields for Automotive Radar Point Cloudscited by 7
  27. 2025VGD: Visual Geometry Gaussian Splatting for Feed-Forward Surround-view Driving Reconstructioncited by 7
  28. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5
  29. 2025OmniGen: Unified Multimodal Sensor Generation for Autonomous Drivingcited by 5
  30. 2024V-RoAst: Visual Road Assessment. Can VLM be a Road Safety Assessor Using the iRAP Standard?cited by 4
  31. 2025GASP: Unifying Geometric and Semantic Self-Supervised Pre-training for Autonomous Drivingcited by 4
  32. 2025PosePilot: Steering Camera Pose for Generative World Models with Self-supervised Depthcited by 4
  33. 2025Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learningcited by 4
  34. 2025SQS: Enhancing Sparse Perception Models via Query-based Splatting in Autonomous Drivingcited by 4
  35. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  36. 2025PSA-SSL: Pose and Size-aware Self-Supervised Learning on LiDAR Point Cloudscited by 3
  37. 2026HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generationcited by 3
  38. 2024TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perceptioncited by 2

50 most cited of 73 All 73 (JSON)