mlsanityData to 14 Oct 2026

Point2Vec for Self-Supervised Representation Learning on Point Clouds

arXiv 2303.16570 ↗First posted 29 Mar 2023cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

Recently, the self-supervised learning framework data2vec has shown inspiring performance for various modalities using a masked student-teacher approach. However, it remains open whether such a framework generalizes to the unique challenges of 3D point clouds. To answer this question, we extend data2vec to the point cloud domain and report encouraging results on several downstream tasks. In an in-depth analysis, we discover that the leakage of positional information reveals the overall object shape to the student even under heavy masking and thus hampers data2vec to learn strong representations for point clouds. We address this 3D-specific shortcoming by proposing point2vec, which unleashes the full potential of data2vec-like pre-training on point clouds. Our experiments show that point2vec outperforms other self-supervised methods on shape classification and few-shot learning on ModelNet40 and ScanObjectNN, while achieving competitive results on part segmentation on ShapeNetParts. These results suggest that the learned representations are strong and transferable, highlighting point2vec as a promising direction for self-supervised learning of point cloud representations.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 45 of its 54 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

45 of 54 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  11. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  12. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
33 more
  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  8. 2019Contrastive Multiview Codingcited by 1,301
  9. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  11. 2021Multiscale Vision Transformerscited by 897
  12. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  13. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  14. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  15. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  16. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  17. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  18. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  19. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  20. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  21. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  22. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 379
  23. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
  24. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  25. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  26. 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207
  27. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  28. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  29. 2022Masked Discrimination for Self-Supervised Learning on Point Cloudscited by 155
  30. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 144
  31. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 122
  32. 2020Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Cloudscited by 75
  33. 2020Self-Supervised Few-Shot Learning on Point Cloudscited by 66

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 10; 2025: 10; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 137
  2. 2024Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Modelcited by 57
  3. 2024Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloudcited by 30
  4. 20243D-JEPA: A Joint Embedding Predictive Architecture for 3D Self-Supervised Representation Learningcited by 14
  5. 2024CloudFixer: Test-Time Adaptation for 3D Point Clouds via Diffusion-Guided Geometric Transformationcited by 11
  6. 2024ConDense: Consistent 2D/3D Pre-training for Dense and Sparse Features from Multi-View Imagescited by 9
  7. 2024CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objectscited by 8
  8. 2024Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Modelscited by 7
  9. 2025Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Cloudscited by 7
  10. 2025Multispectral airborne laser scanning for tree species classification: a benchmark of machine learning and deep learning algorithmscited by 6
  11. 2023Group Multi-View Transformer for 3D Shape Analysis with Spatial Encodingcited by 4
  12. 2025CrossJEPA: Cross-Modal Joint-Embedding Predictive Architecture for Efficient 3D Representation Learning from 2D Imagescited by 4
16 more
  1. 2026Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understandingcited by 4
  2. 2025Implicit Neural-Representation Learning for Elastic Deformable-Object Manipulationscited by 2
  3. 2026Emerging Extrinsic Dexterity in Cluttered Scenes via Dynamics-aware Policy Learningcited by 2
  4. 2024Point-PNG: Conditional Pseudo-Negatives Generation for Point Cloud Pre-Trainingcited by 1
  5. 2025Foundry: Distilling 3D Foundation Models for the Edgecited by 1
  6. 2026Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physicscited by 1
  7. 2023M3CS: Multi-Target Masked Point Modeling with Learnable Codebook and Siamese Decoderscited by 0
  8. 2024Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentationcited by 0
  9. 2025Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learningcited by 0
  10. 2025PointGAC: Geometric-Aware Codebook for Masked Point Cloud Modelingcited by 0
  11. 2025Deformable Cluster Manipulation via Whole-Arm Policy Learningcited by 0
  12. 2025ModelNet40-E: An Uncertainty-Aware Benchmark for Point Cloud Classificationcited by 0
  13. 2025Label-Efficient Grasp Joint Prediction with Point-JEPAcited by 0
  14. 2026FILTR: Extracting Topological Features from Pretrained 3D Modelscited by 0
  15. 2026SR-JEPA: Learning Predictive Latent State in 3D Scenescited by 0
  16. 2026Synthetic LiDAR Data Generation and Deterministic Downsampling for Point Cloud Classification on the Edgecited by 0