mlsanityData to 7 Oct 2026

A Closer Look at Invariances in Self-supervised Pre-training for 3D Vision

arXiv 2207.04997 ↗First posted 11 Jul 2022cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

Self-supervised pre-training for 3D vision has drawn increasing research interest in recent years. In order to learn informative representations, a lot of previous works exploit invariances of 3D features, e.g., perspective-invariance between views of the same scene, modality-invariance between depth and RGB images, format-invariance between point clouds and voxels. Although they have achieved promising results, previous researches lack a systematic and fair comparison of these invariances. To address this issue, our work, for the first time, introduces a unified framework, under which various pre-training methods can be investigated. We conduct extensive experiments and provide a closer look at the contributions of different invariances in 3D pre-training. Also, we propose a simple but effective method that jointly pre-trains a 3D encoder and a depth map encoder using contrastive learning. Models pre-trained with our method gain significant performance boost in downstream tasks. For instance, a pre-trained VoteNet outperforms previous methods on SUN RGB-D and ScanNet object detection benchmarks with a clear margin.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 50 of its 72 references that are in the map. First posted on arXiv 11 Jul 2022.

Builds on

50 of 72 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
38 more
  1. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  2. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  3. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  4. 2016Colorful Image Colorizationcited by 1,113
  5. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  6. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  7. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  8. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  9. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  10. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  11. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  12. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  13. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  14. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  15. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  16. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  17. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  18. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  19. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  20. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  21. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  22. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  23. 2021Deep Parametric Continuous Convolutional Neural Networkscited by 161
  24. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  25. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  26. 20183DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentationcited by 118
  27. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  28. 2016Vehicle Detection from 3D Lidar Using Fully Convolutional Networkcited by 105
  29. 2019Unsupervised Multi-Task Feature Learning on Point Cloudscited by 89
  30. 2020Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detectioncited by 78
  31. 2020Info3D: Representation Learning on 3D Objects using Mutual Information Maximization and Contrastive Learningcited by 68
  32. 2020Cross-Modal Weighting Network for RGB-D Salient Object Detectioncited by 47
  33. 20213D-to-2D Distillation for Indoor Scene Parsingcited by 47
  34. 2021Contrastive Multimodal Fusion with TupleInfoNCEcited by 43
  35. 2020P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understandingcited by 41
  36. 2020RangeRCNN: Towards Fast and Accurate 3D Object Detection with Range Image Representationcited by 37
  37. 2021Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretrainingcited by 32
  38. 2020Malleable 2.5D Convolution: Learning Receptive Fields along the Depth-axis for RGB-D Scene Parsingcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 5; 2025: 2; 2026: 3.

24 papers.

Sort:

  1. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  2. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 123
  3. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  4. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  5. 2022Ponder: Point Cloud Pre-training via Neural Renderingcited by 50
  6. 2023Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Losscited by 35
  7. 2023Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Modelscited by 26
  8. 2023Self-supervised Learning for Pre-Training 3D Point Clouds: A Surveycited by 21
  9. 2023Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understandingcited by 13
  10. 2023T-MAE: Temporal Masked Autoencoders for Point Cloud Representation Learningcited by 11
  11. 2025LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Drivingcited by 11
  12. 2024Shelf-Supervised Cross-Modal Pre-Training for 3D Object Detectioncited by 9
12 more
  1. 2024MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learningcited by 8
  2. 2024Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Modelscited by 5
  3. 2023Point Clouds Are Specialized Images: A Knowledge Transfer Approach for 3D Understandingcited by 3
  4. 2024SAM-Guided Masked Token Prediction for 3D Scene Understandingcited by 3
  5. 2026A Mixed Diet Makes DINO An Omnivorous Vision Encodercited by 3
  6. 2023Applying Plain Transformers to Real-World Point Cloudscited by 2
  7. 2023Point Cloud Self-supervised Learning via 3D to Multi-view Masked Learnercited by 2
  8. 2026PointCSP: Cross-Sample Semantic Propagation and Stability Preservation in Self-Supervised Point Cloud Learningcited by 1
  9. 2023Randomized 3D Scene Generation for Generalizable Self-Supervised Pre-Trainingcited by 0
  10. 2024Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understandingcited by 0
  11. 2025Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoderscited by 0
  12. 2026RoAD Benchmark: How LiDAR Models Fail under Coupled Domain Shifts and Label Evolutioncited by 0