mlsanityData to 7 Oct 2026

Masked Autoencoders for Point Cloud Self-supervised Learning

arXiv 2203.06604 ↗First posted 13 Mar 2022cs.CVcited by 456

Not on the map. Why?

Abstract (arXiv)

As a promising scheme of self-supervised learning, masked autoencoding has significantly advanced natural language processing and computer vision. Inspired by this, we propose a neat scheme of masked autoencoders for point cloud self-supervised learning, addressing the challenges posed by point cloud's properties, including leakage of location information and uneven information density. Concretely, we divide the input point cloud into irregular point patches and randomly mask them at a high ratio. Then, a standard Transformer based autoencoder, with an asymmetric design and a shifting mask tokens operation, learns high-level latent features from unmasked point patches, aiming to reconstruct the masked point patches. Extensive experiments show that our approach is efficient during pre-training and generalizes well on various downstream tasks. Specifically, our pre-trained models achieve 85.18% accuracy on ScanObjectNN and 94.04% accuracy on ModelNet40, outperforming all the other self-supervised learning methods. We show with our scheme, a simple architecture entirely based on standard Transformers can surpass dedicated Transformer models from supervised learning. Our approach also advances state-of-the-art accuracies by 1.5%-2.3% in the few-shot object classification. Furthermore, our work inspires the feasibility of applying unified architectures from languages and images to the point cloud.

Key facts. Cited by 456 papers in the map (data through 7 Oct 2026). Builds on 51 of its 63 references that are in the map. First posted on arXiv 13 Mar 2022.

Builds on

51 of 63 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
38 more
  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  5. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  6. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  9. 2020Point Transformercited by 1,462
  10. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  11. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  12. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  13. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  14. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  15. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  16. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  17. 2020PCT: Point cloud transformercited by 730
  18. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  19. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  20. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  21. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  22. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  23. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  24. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  25. 2019Relation-Shape Convolutional Neural Network for Point Cloud Analysiscited by 346
  26. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  27. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  28. 2018SO-Net: Self-Organizing Network for Point Cloud Analysiscited by 210
  29. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  30. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  31. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 164
  32. 2021CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attentioncited by 139
  33. 2016Discrete Variational Autoencoderscited by 124
  34. 2020Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Cloudscited by 75
  35. 2020Self-Supervised Few-Shot Learning on Point Cloudscited by 66
  36. 2021PRA-Net: Point Relation-Aware Network for 3D Point Cloud Analysiscited by 58
  37. 2022Implicit Autoencoder for Point-Cloud Self-Supervised Representation Learningcited by 46
  38. 2022HiP: Hierarchical Perceivercited by 15

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 36; 2023: 124; 2024: 116; 2025: 105; 2026: 75.

456 papers.

Sort:

By area:

  1. 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
  2. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  3. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  4. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  5. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  6. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  7. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  8. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  9. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  10. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 183
  11. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  12. 2024PointMamba: A Simple State Space Model for Point Cloud Analysiscited by 166
38 more
  1. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 162
  2. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  3. 2022Masked Discrimination for Self-Supervised Learning on Point Cloudscited by 155
  4. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 143
  5. 2023Multimodal Industrial Anomaly Detection via Hybrid Fusioncited by 138
  6. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  7. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  8. 2022Vision-Centric BEV Perception: A Surveycited by 126
  9. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  10. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  11. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 121
  12. 2023PointGPT: Auto-regressively Generative Pre-training from Point Cloudscited by 115
  13. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  14. 2023Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understandingcited by 111
  15. 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 108
  16. 2023OctFormer: Octree-based Transformers for 3D Point Cloudscited by 105
  17. 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 92
  18. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  19. 2023Real3D-AD: A Dataset of Point Cloud Anomaly Detectioncited by 87
  20. 2023CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognitioncited by 77
  21. 2022P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Promptingcited by 74
  22. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  23. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  24. 2023CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Datacited by 69
  25. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generationcited by 69
  26. 2022ALSO: Automotive Lidar Self-supervision by Occupancy estimationcited by 68
  27. 2023Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Modelscited by 67
  28. 2023Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Networkcited by 67
  29. 2024Point Cloud Mamba: Point Cloud Learning via State Space Modelcited by 66
  30. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  31. 2025Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awarenesscited by 61
  32. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 60
  33. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  34. 2022Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoderscited by 57
  35. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  36. 2023Hard Patches Mining for Masked Image Modelingcited by 56
  37. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  38. 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53

50 most cited of 456 All 456 (JSON)