mlsanityData to 14 Oct 2026

Pre-Training LiDAR-Based 3D Object Detectors Through Colorization

arXiv 2310.14592 ↗First posted 23 Oct 2023cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Accurate 3D object detection and understanding for self-driving cars heavily relies on LiDAR point clouds, necessitating large amounts of labeled data to train. In this work, we introduce an innovative pre-training approach, Grounded Point Colorization (GPC), to bridge the gap between data and labels by teaching the model to colorize LiDAR point clouds, equipping it with valuable semantic cues. To tackle challenges arising from color variations and selection bias, we incorporate color as "context" by providing ground-truth colors as hints during colorization. Experimental results on the KITTI and Waymo datasets demonstrate GPC's remarkable effectiveness. Even with limited labeled data, GPC significantly improves fine-tuning performance; notably, on just 20% of the KITTI dataset, GPC outperforms training from scratch with the entire dataset. In sum, we introduce a fresh perspective on pre-training for 3D object detection, aligning the objective with the model's intended role and ultimately advancing the accuracy and efficiency of 3D object detection for autonomous vehicles.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 61 of its 89 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

61 of 89 references.

References · 61

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  10. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  11. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
38 more
  1. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  3. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  6. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  7. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  8. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  9. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  10. 2016Colorful Image Colorizationcited by 1,114
  11. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  12. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  13. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  14. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  15. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  16. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  17. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  18. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  19. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  20. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  21. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  22. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 400
  23. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  24. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  25. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 280
  26. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  27. 2016Learning Representations for Automatic Colorizationcited by 251
  28. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  29. 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
  30. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  31. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  32. 2017Colorization as a Proxy Task for Visual Understandingcited by 175
  33. 2022Masked Discrimination for Self-Supervised Learning on Point Cloudscited by 155
  34. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  35. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  36. 2021Colorization Transformercited by 89
  37. 2022Fully Sparse 3D Object Detectioncited by 84
  38. 2018Deep Exemplar-based Colorizationcited by 82

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2025Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scenecited by 5
  2. 2024Learning 3D Perception from Others' Predictionscited by 4
  3. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  4. 2025PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detectioncited by 1
  5. 2024Finetuning Pre-trained Model with Limited Data for LiDAR-based 3D Object Detection by Bridging Domain Gapscited by 0