mlsanityData to 7 Oct 2026

PRED: Pre-training via Semantic Rendering on LiDAR Point Clouds

arXiv 2311.04501 ↗First posted 8 Nov 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Pre-training is crucial in 3D-related fields such as autonomous driving where point cloud annotation is costly and challenging. Many recent studies on point cloud pre-training, however, have overlooked the issue of incompleteness, where only a fraction of the points are captured by LiDAR, leading to ambiguity during the training phase. On the other hand, images offer more comprehensive information and richer semantics that can bolster point cloud encoders in addressing the incompleteness issue inherent in point clouds. Yet, incorporating images into point cloud pre-training presents its own challenges due to occlusions, potentially causing misalignments between points and pixels. In this work, we propose PRED, a novel image-assisted pre-training framework for outdoor point clouds in an occlusion-aware manner. The main ingredient of our framework is a Birds-Eye-View (BEV) feature map conditioned semantic rendering, leveraging the semantics of images for supervision through neural rendering. We further enhance our model's performance by incorporating point-wise masking with a high mask ratio (95%). Extensive experiments demonstrate PRED's superiority over prior point cloud pre-training methods, providing significant improvements on various large-scale datasets for 3D perception tasks. Codes will be available at https://github.com/PRED4pc/PRED.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 60 of its 65 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

60 of 65 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  10. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  12. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
38 more
  1. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  2. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  3. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  4. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  5. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  6. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  7. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  8. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  9. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  10. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  11. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  12. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  13. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  14. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 409
  15. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  16. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  17. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  18. 2021Voxel Transformer for 3D Object Detectioncited by 247
  19. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
  20. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  21. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  22. 2021Group-Free 3D Object Detection via Transformerscited by 215
  23. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  24. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  25. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
  26. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  27. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  28. 2023DSVT: Dynamic Sparse Voxel Transformer with Rotated Setscited by 122
  29. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  30. 2021Light Field Neural Renderingcited by 108
  31. 2022PillarNet: Real-Time and High-Performance Pillar-based 3D Object Detectioncited by 99
  32. 2022SWFormer: Sparse Window Transformer for 3D Object Detection in Point Cloudscited by 92
  33. 2021M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformerscited by 74
  34. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  35. 2021Back-tracing Representative Points for Voting-based 3D Object Detection in Point Cloudscited by 67
  36. 2022CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Cloudscited by 58
  37. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  38. 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 4.

7 papers.

  1. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  2. 2025DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Drivingcited by 20
  3. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19
  4. 2025TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Predictioncited by 9
  5. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  6. 2024Point Cloud Unsupervised Pre-training via 3D Gaussian Splattingcited by 1
  7. 2025Self-Supervised Pre-training with Combined Datasets for 3D Perception in Autonomous Drivingcited by 0