mlsanityData to 14 Oct 2026

Pic@Point: Cross-Modal Learning by Local and Global Point-Picture Correspondence

arXiv 2410.09519 ↗First posted 12 Oct 2024cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Self-supervised pre-training has achieved remarkable success in NLP and 2D vision. However, these advances have yet to translate to 3D data. Techniques like masked reconstruction face inherent challenges on unstructured point clouds, while many contrastive learning tasks lack in complexity and informative value. In this paper, we present Pic@Point, an effective contrastive learning method based on structural 2D-3D correspondences. We leverage image cues rich in semantic and contextual knowledge to provide a guiding signal for point cloud representations at various abstraction levels. Our lightweight approach outperforms state-of-the-art pre-training methods on several 3D benchmarks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 49 references that are in the map. First posted on arXiv 12 Oct 2024.

Builds on

39 of 49 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  10. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  11. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  12. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
27 more
  1. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  2. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  3. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  4. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  5. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 565
  6. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  7. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  8. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  9. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
  10. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 379
  11. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 342
  12. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  13. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  14. 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
  15. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  16. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  17. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 164
  18. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 144
  19. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 122
  20. 2021Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completioncited by 112
  21. 2020Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Cloudscited by 75
  22. 2022P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Promptingcited by 74
  23. 2018View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictionscited by 43
  24. 2021Image2Point: 3D Point-Cloud Understanding with 2D Image Pretrained Modelscited by 41
  25. 2023Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Modelscited by 25
  26. 2022Self-Supervised Learning with Multi-View Rendering for 3D Point Cloud Analysiscited by 8
  27. 2022PointVST: Self-Supervised Pre-training for 3D Point Clouds via View-Specific Point-to-Image Translationcited by 7

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025CrossJEPA: Cross-Modal Joint-Embedding Predictive Architecture for Efficient 3D Representation Learning from 2D Imagescited by 4