mlsanityData to 7 Oct 2026

Masked Discrimination for Self-Supervised Learning on Point Clouds

arXiv 2203.11183 ↗First posted 21 Mar 2022cs.CVcited by 155

Not on the map. Why?

Abstract (arXiv)

Masked autoencoding has achieved great success for self-supervised learning in the image and language domains. However, mask based pretraining has yet to show benefits for point cloud understanding, likely due to standard backbones like PointNet being unable to properly handle the training versus testing distribution mismatch introduced by masking during training. In this paper, we bridge this gap by proposing a discriminative mask pretraining Transformer framework, MaskPoint}, for point clouds. Our key idea is to represent the point cloud as discrete occupancy values (1 if part of the point cloud; 0 if not), and perform simple binary classification between masked object points and sampled noise points as the proxy task. In this way, our approach is robust to the point sampling variance in point clouds, and facilitates learning rich representations. We evaluate our pretrained models across several downstream tasks, including 3D shape classification, segmentation, and real-word object detection, and demonstrate state-of-the-art results while achieving a significant pretraining speedup (e.g., 4.1x on ScanNet) compared to the prior state-of-the-art Transformer baseline. Code is available at https://github.com/haotian-liu/MaskPoint.

Key facts. Cited by 155 papers in the map (data through 7 Oct 2026). Builds on 65 of its 75 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

65 of 75 references.

References · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  10. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  11. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  12. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
38 more
  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  2. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  7. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  8. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  10. 2020Point Transformercited by 1,462
  11. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  13. 2017Random Erasing Data Augmentationcited by 1,235
  14. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  15. 2016Colorful Image Colorizationcited by 1,113
  16. 2016Deep Networks with Stochastic Depthcited by 973
  17. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  18. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  19. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  20. 2020PCT: Point cloud transformercited by 730
  21. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  22. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  23. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  24. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  25. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  26. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  27. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  28. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  29. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  30. 2021Early Convolutions Help Transformers See Bettercited by 395
  31. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  32. 2019Relation-Shape Convolutional Neural Network for Point Cloud Analysiscited by 346
  33. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  34. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  35. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  36. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  37. 2018DropBlock: A regularization method for convolutional networkscited by 261
  38. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 17; 2023: 53; 2024: 37; 2025: 37; 2026: 11.

155 papers.

Sort:

By area:

  1. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  2. 2024PointMamba: A Simple State Space Model for Point Cloud Analysiscited by 166
  3. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 162
  4. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 143
  5. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 121
  6. 2023PointGPT: Auto-regressively Generative Pre-training from Point Cloudscited by 115
  7. 2023Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understandingcited by 111
  8. 2023RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Drivingcited by 76
  9. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  10. 2023CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Datacited by 69
  11. 2023Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Modelscited by 67
  12. 2024Point Cloud Mamba: Point Cloud Learning via State Space Modelcited by 66
38 more
  1. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  2. 2022Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoderscited by 57
  3. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  4. 2024Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Modelcited by 57
  5. 2023Hard Patches Mining for Masked Image Modelingcited by 56
  6. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  7. 2023Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-trainingcited by 52
  8. 2024Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysiscited by 52
  9. 2022Ponder: Point Cloud Pre-training via Neural Renderingcited by 50
  10. 2023Point Cloud Pre-training with Diffusion Modelscited by 47
  11. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46
  12. 2024Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learningcited by 43
  13. 2023Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoderscited by 42
  14. 2022Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Cloudscited by 39
  15. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generationcited by 37
  16. 2022Masked Autoencoders in 3D Point Cloud Representation Learningcited by 30
  17. 2023Point2Vec for Self-Supervised Representation Learning on Point Cloudscited by 28
  18. 2024Masked Image Modeling: A Surveycited by 28
  19. 2023MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Trainingcited by 26
  20. 2023Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Modelscited by 26
  21. 2023Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Modelscited by 25
  22. 2023Advancements in Point Cloud Data Augmentation for Deep Learning: A Surveycited by 24
  23. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  24. 2024PCP-MAE: Learning to Predict Centers for Point Masked Autoencoderscited by 23
  25. 2023PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videoscited by 22
  26. 2023Self-supervised Learning for Pre-Training 3D Point Clouds: A Surveycited by 21
  27. 2022MAELi: Masked Autoencoder for Large-Scale LiDAR Point Cloudscited by 20
  28. 2024NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fieldscited by 20
  29. 2023P2C: Self-Supervised Point Cloud Completion from Single Partial Cloudscited by 19
  30. 2023Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videoscited by 19
  31. 20223D-OAE: Occlusion Auto-Encoders for Self-Supervised Learning on Point Cloudscited by 18
  32. 2022EPCL: Frozen CLIP Transformer is An Efficient Point Cloud Encodercited by 18
  33. 2023ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentationcited by 18
  34. 2024Parameter-efficient Prompt Learning for 3D Point Cloud Understandingcited by 16
  35. 2024Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learningcited by 16
  36. 2022Masked Surfel Prediction for Self-Supervised Point Cloud Learningcited by 15
  37. 2023Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrastcited by 14
  38. 2023VPP: Efficient Conditional 3D Generation via Voxel-Point Progressive Representationcited by 14

50 most cited of 155 All 155 (JSON)