mlsanityData to 7 Oct 2026

Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

arXiv 2302.02318 ↗First posted 5 Feb 2023cs.CVcited by 162

Not on the map. Why?

Abstract (arXiv)

Mainstream 3D representation learning approaches are built upon contrastive or generative modeling pretext tasks, where great improvements in performance on various downstream tasks have been achieved. However, we find these two paradigms have different characteristics: (i) contrastive models are data-hungry that suffer from a representation over-fitting issue; (ii) generative models have a data filling issue that shows inferior data scaling capacity compared to contrastive models. This motivates us to learn 3D representations by sharing the merits of both paradigms, which is non-trivial due to the pattern difference between the two paradigms. In this paper, we propose Contrast with Reconstruct (ReCon) that unifies these two paradigms. ReCon is trained to learn from both generative modeling teachers and single/cross-modal contrastive teachers through ensemble distillation, where the generative student guides the contrastive student. An encoder-decoder style ReCon-block is proposed that transfers knowledge through cross attention with stop-gradient, which avoids pretraining over-fitting and pattern difference issues. ReCon achieves a new state-of-the-art in 3D representation learning, e.g., 91.26% accuracy on ScanObjectNN. Codes have been released at https://github.com/qizekun/ReCon.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). Builds on 82 of its 107 references that are in the map. First posted on arXiv 5 Feb 2023.

Builds on

82 of 107 references.

References · 82

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  8. 2022Training language models to follow instructions with human feedbackcited by 15,712
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  11. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  12. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
38 more
  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  13. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  14. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  15. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  16. 2020Supervised Contrastive Learningcited by 2,803
  17. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  18. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  19. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  20. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  21. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  22. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  23. 2019Contrastive Multiview Codingcited by 1,297
  24. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  25. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  26. 2021Grounded Language-Image Pre-trainingcited by 1,147
  27. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  28. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  29. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  30. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  31. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  32. 2019Contrastive Representation Distillationcited by 734
  33. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  34. 2020PCT: Point cloud transformercited by 730
  35. 2018MINE: Mutual Information Neural Estimationcited by 680
  36. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  37. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  38. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 27; 2024: 49; 2025: 51; 2026: 35.

162 papers.

Sort:

By area:

  1. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  2. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  3. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  4. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  5. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  6. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 183
  7. 2024PointMamba: A Simple State Space Model for Point Cloud Analysiscited by 166
  8. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  9. 2023PointGPT: Auto-regressively Generative Pre-training from Point Cloudscited by 115
  10. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 105
  11. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  12. 2023Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Modelscited by 67
38 more
  1. 2024Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Modelcited by 57
  2. 2024Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysiscited by 52
  3. 2025SceneSplat: Gaussian Splatting-based Scene Understanding with Vision-Language Pretrainingcited by 51
  4. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  5. 2023OmniVec: Learning robust representations with cross modal sharingcited by 44
  6. 2024Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learningcited by 43
  7. 2023Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoderscited by 42
  8. 2023Connecting Multi-modal Contrastive Representationscited by 36
  9. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 34
  10. 2024TAMM: TriAdapter Multi-Modal Learning for 3D Shape Understandingcited by 26
  11. 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
  12. 2023Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Modelscited by 25
  13. 2023DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classificationcited by 23
  14. 2024Any2Point: Empowering Any-modality Large Models for Efficient 3D Understandingcited by 23
  15. 2024PCP-MAE: Learning to Predict Centers for Point Masked Autoencoderscited by 23
  16. 2023DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Taskscited by 22
  17. 2024PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learningcited by 21
  18. 2024ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretrainingcited by 21
  19. 2025URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Modelcited by 20
  20. 2023Prompted Contrast with Masked Motion Modeling: Towards Versatile 3D Action Representation Learningcited by 19
  21. 2025DexVLG: Dexterous Vision-Language-Grasp Model at Scalecited by 19
  22. 2024UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learningcited by 18
  23. 2024SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Mannerscited by 18
  24. 2025Reasoning in Space via Grounding in the Worldcited by 18
  25. 2026Utonia: Toward One Encoder for All Point Cloudscited by 18
  26. 2023Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-trainingcited by 17
  27. 2025OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Modelcited by 17
  28. 2024Parameter-efficient Prompt Learning for 3D Point Cloud Understandingcited by 16
  29. 2024Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learningcited by 16
  30. 2024One Leaf Reveals the Season: Occlusion-Based Contrastive Learning with Semantic-Aware Views for Efficient Visual Representationcited by 16
  31. 20253UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understandingcited by 16
  32. 2025Exploring the Potential of Encoder-free Architectures in 3D LMMscited by 16
  33. 20233DAxiesPrompts: Unleashing the 3D Spatial Task Capabilities of GPT-4Vcited by 15
  34. 2024Positional Prompt Tuning for Efficient 3D Representation Learningcited by 15
  35. 2024RI-MAE: Rotation-Invariant Masked AutoEncoders for Self-Supervised Point Cloud Representation Learningcited by 15
  36. 2023Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrastcited by 14
  37. 2023VPP: Efficient Conditional 3D Generation via Voxel-Point Progressive Representationcited by 14
  38. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13

50 most cited of 162 All 162 (JSON)