mlsanityData to 14 Oct 2026

4DContrast: Contrastive Learning with Dynamic Correspondences for 3D Scene Understanding

arXiv 2112.02990 ↗First posted 6 Dec 2021cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

We present a new approach to instill 4D dynamic object priors into learned 3D representations by unsupervised pre-training. We observe that dynamic movement of an object through an environment provides important cues about its objectness, and thus propose to imbue learned 3D representations with such dynamic understanding, that can then be effectively transferred to improved performance in downstream 3D semantic scene understanding tasks. We propose a new data augmentation scheme leveraging synthetic 3D shapes moving in static 3D environments, and employ contrastive learning under 3D-4D constraints that encode 4D invariances into the learned 3D representations. Experiments demonstrate that our unsupervised representation learning results in improvement in downstream 3D semantic segmentation, object detection, and instance segmentation tasks, and moreover, notably improves performance in data-scarce scenarios.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 41 of its 52 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

41 of 52 references.

References · 41

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  8. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  9. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  10. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  11. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  12. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
29 more
  1. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  2. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 565
  3. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  4. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  5. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 321
  6. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 217
  7. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  8. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  9. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  10. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  11. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  12. 2018GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloudcited by 146
  13. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 141
  14. 2021Mix3D: Out-of-Context Data Augmentation for 3D Scenescited by 126
  15. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  16. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  17. 20183DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentationcited by 118
  18. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  19. 2020Virtual Multi-view Fusion for 3D Semantic Segmentationcited by 99
  20. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95
  21. 2019Unsupervised Multi-Task Feature Learning on Point Cloudscited by 89
  22. 2020MLCVNet: Multi-Level Context VoteNet for 3D Object Detectioncited by 80
  23. 2020Info3D: Representation Learning on 3D Objects using Mutual Information Maximization and Contrastive Learningcited by 68
  24. 2019Point Cloud Instance Segmentation using Probabilistic Embeddingscited by 52
  25. 2018TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshescited by 45
  26. 2020DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshescited by 42
  27. 2021RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detectioncited by 37
  28. 2020RfD-Net: Point Scene Understanding by Semantic Instance Reconstructioncited by 36
  29. 2020Unsupervised 3D Learning for Shape Analysis via Multiresolution Instance Discriminationcited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 11; 2023: 21; 2024: 13; 2025: 8; 2026: 1.

54 papers.

Sort:

By area:

  1. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
  2. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 164
  3. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 125
  4. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 122
  5. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  6. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  7. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  8. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  9. 2022Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Surveycited by 54
  10. 2022Ponder: Point Cloud Pre-training via Neural Renderingcited by 50
  11. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  12. 20244D Contrastive Superflows are Dense 3D Representation Learnerscited by 33
38 more
  1. 2022Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learningcited by 30
  2. 2023Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Modelscited by 27
  3. 2023Self-supervised Learning for Pre-Training 3D Point Clouds: A Surveycited by 21
  4. 2023SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstructioncited by 21
  5. 2023A Survey of Label-Efficient Deep Learning for 3D Point Cloudscited by 20
  6. 2023Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariancecited by 19
  7. 2023Lang3DSG: Language-based contrastive pre-training for 3D Scene Graph predictioncited by 19
  8. 2022COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentationcited by 15
  9. 2023Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priorscited by 15
  10. 2022PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Datacited by 14
  11. 2023Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrastcited by 14
  12. 2022MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistencycited by 12
  13. 2025LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Drivingcited by 12
  14. 2022Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learningcited by 9
  15. 2024MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learningcited by 8
  16. 2024Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Datacited by 8
  17. 2023SSR-2D: Semantic 3D Scene Reconstruction from 2D Imagescited by 7
  18. 2024Self-supervised Learning of LiDAR 3D Point Clouds via 2D-3D Neural Calibrationcited by 7
  19. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  20. 2023Generalizable Local Feature Pre-training for Deformable Shape Analysiscited by 6
  21. 2024DHGCN: Dynamic Hop Graph Convolution Network for Self-Supervised Point Cloud Learningcited by 6
  22. 2024CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understandingcited by 6
  23. 2024Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchangecited by 5
  24. 2024Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Modelscited by 5
  25. 2025Enhanced Spatiotemporal Consistency for Image-to-LiDAR Data Pretrainingcited by 4
  26. 2025Masked Scene Modeling: Narrowing the Gap Between Supervised and Self-Supervised Learning in 3D Scene Understandingcited by 4
  27. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  28. 2022SeRP: Self-Supervised Representation Learning Using Perturbed Point Cloudscited by 3
  29. 2023Multi-View Representation is What You Need for Point-Cloud Pre-Trainingcited by 3
  30. 2024Hyperbolic Image-and-Pointcloud Contrastive Learning for 3D Classificationcited by 3
  31. 2024SAM-Guided Masked Token Prediction for 3D Scene Understandingcited by 3
  32. 2023Applying Plain Transformers to Real-World Point Cloudscited by 2
  33. 2023Shape Anchor Guided Holistic Indoor Scene Understandingcited by 2
  34. 2023M33D: Learning 3D priors using Multi-Modal Masked Autoencoders for 2D image and video understandingcited by 2
  35. 2024EPContrast: Effective Point-level Contrastive Learning for Large-scale Point Cloud Understandingcited by 2
  36. 2025Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understandingcited by 2
  37. 2023EipFormer: Emphasizing Instance Positions in 3D Instance Segmentationcited by 1
  38. 2024Point Cloud Unsupervised Pre-training via 3D Gaussian Splattingcited by 1

50 most cited of 54 All 54 (JSON)