# Point2Vec for Self-Supervised Representation Learning on Point Clouds

arXiv: https://arxiv.org/abs/2303.16570 · HTML: https://mlsanity.com/paper/2303.16570/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 54 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 30 of 45)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)

## Built on by (most cited first, 28 of 28)

- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model](https://mlsanity.com/paper/2404.14966/) (2024)
- [Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud](https://mlsanity.com/paper/2404.16432/) (2024)
- [3D-JEPA: A Joint Embedding Predictive Architecture for 3D Self-Supervised Representation Learning](https://mlsanity.com/paper/2409.15803/) (2024)
- [CloudFixer: Test-Time Adaptation for 3D Point Clouds via Diffusion-Guided Geometric Transformation](https://mlsanity.com/paper/2407.16193/) (2024)
- [ConDense: Consistent 2D/3D Pre-training for Dense and Sparse Features from Multi-View Images](https://mlsanity.com/paper/2408.17027/) (2024)
- [CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objects](https://mlsanity.com/paper/2403.10760/) (2024)
- [Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Models](https://mlsanity.com/paper/2408.06145/) (2024)
- [Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds](https://mlsanity.com/paper/2502.20316/) (2025)
- [Multispectral airborne laser scanning for tree species classification: a benchmark of machine learning and deep learning algorithms](https://mlsanity.com/paper/2504.14337/) (2025)
- [Group Multi-View Transformer for 3D Shape Analysis with Spatial Encoding](https://mlsanity.com/paper/2312.16477/) (2023)
- [CrossJEPA: Cross-Modal Joint-Embedding Predictive Architecture for Efficient 3D Representation Learning from 2D Images](https://mlsanity.com/paper/2511.18424/) (2025)
- [Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding](https://mlsanity.com/paper/2604.19609/) (2026)
- [Implicit Neural-Representation Learning for Elastic Deformable-Object Manipulations](https://mlsanity.com/paper/2505.00500/) (2025)
- [Emerging Extrinsic Dexterity in Cluttered Scenes via Dynamics-aware Policy Learning](https://mlsanity.com/paper/2603.09882/) (2026)
- [Point-PNG: Conditional Pseudo-Negatives Generation for Point Cloud Pre-Training](https://mlsanity.com/paper/2409.15832/) (2024)
- [Foundry: Distilling 3D Foundation Models for the Edge](https://mlsanity.com/paper/2511.20721/) (2025)
- [Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics](https://mlsanity.com/paper/2609.00611/) (2026)
- [M3CS: Multi-Target Masked Point Modeling with Learnable Codebook and Siamese Decoders](https://mlsanity.com/paper/2309.13235/) (2023)
- [Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentation](https://mlsanity.com/paper/2407.08994/) (2024)
- [Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning](https://mlsanity.com/paper/2506.21724/) (2025)
- [PointGAC: Geometric-Aware Codebook for Masked Point Cloud Modeling](https://mlsanity.com/paper/2507.04801/) (2025)
- [Deformable Cluster Manipulation via Whole-Arm Policy Learning](https://mlsanity.com/paper/2507.17085/) (2025)
- [ModelNet40-E: An Uncertainty-Aware Benchmark for Point Cloud Classification](https://mlsanity.com/paper/2508.01269/) (2025)
- [Label-Efficient Grasp Joint Prediction with Point-JEPA](https://mlsanity.com/paper/2509.13349/) (2025)
- [FILTR: Extracting Topological Features from Pretrained 3D Models](https://mlsanity.com/paper/2604.22334/) (2026)
- [SR-JEPA: Learning Predictive Latent State in 3D Scenes](https://mlsanity.com/paper/2608.05774/) (2026)
- [Synthetic LiDAR Data Generation and Deterministic Downsampling for Point Cloud Classification on the Edge](https://mlsanity.com/paper/2608.07106/) (2026)

## Abstract (arXiv)

Recently, the self-supervised learning framework data2vec has shown inspiring performance for various modalities using a masked student-teacher approach. However, it remains open whether such a framework generalizes to the unique challenges of 3D point clouds. To answer this question, we extend data2vec to the point cloud domain and report encouraging results on several downstream tasks. In an in-depth analysis, we discover that the leakage of positional information reveals the overall object shape to the student even under heavy masking and thus hampers data2vec to learn strong representations for point clouds. We address this 3D-specific shortcoming by proposing point2vec, which unleashes the full potential of data2vec-like pre-training on point clouds. Our experiments show that point2vec outperforms other self-supervised methods on shape classification and few-shot learning on ModelNet40 and ScanObjectNN, while achieving competitive results on part segmentation on ShapeNetParts. These results suggest that the learned representations are strong and transferable, highlighting point2vec as a promising direction for self-supervised learning of point cloud representations.