mlsanityData to 7 Oct 2026

CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understanding

arXiv 2401.09057 ↗First posted 17 Jan 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper introduces a novel approach named CrossVideo, which aims to enhance self-supervised cross-modal contrastive learning in the field of point cloud video understanding. Traditional supervised learning methods encounter limitations due to data scarcity and challenges in label acquisition. To address these issues, we propose a self-supervised learning method that leverages the cross-modal relationship between point cloud videos and image videos to acquire meaningful feature representations. Intra-modal and cross-modal contrastive learning techniques are employed to facilitate effective comprehension of point cloud video. We also propose a multi-level contrastive approach for both modalities. Through extensive experiments, we demonstrate that our method significantly surpasses previous state-of-the-art approaches, and we conduct comprehensive ablation studies to validate the effectiveness of our proposed designs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 33 of its 37 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

33 of 37 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  7. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  8. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  9. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  10. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  11. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  12. 2017Look, Listen and Learncited by 452
21 more
  1. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  2. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  3. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  4. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  5. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  6. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  7. 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
  8. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 192
  9. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  10. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  11. 2021ASFormer: Transformer for Action Segmentationcited by 166
  12. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  13. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecastingcited by 76
  14. 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
  15. 2021Contrastive Self-supervised Sequential Recommendation with Robust Augmentationcited by 56
  16. 20214DContrast: Contrastive Learning with Dynamic Correspondences for 3D Scene Understandingcited by 54
  17. 2020Learning Physical Graph Representations from Visual Scenescited by 50
  18. 2020P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understandingcited by 41
  19. 2021Image2Point: 3D Point-Cloud Understanding with 2D Image Pretrained Modelscited by 41
  20. 2022Point Primitive Transformer for Long-Term 4D Point Cloud Video Understandingcited by 35
  21. 2022Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learningcited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 4.

6 papers.

  1. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  2. 2026Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perceptioncited by 17
  3. 2026Tri-Efficient Transfer Learning for Point Cloud Videoscited by 5
  4. 2024Understanding and Mitigating Human-Labelling Errors in Supervised Contrastive Learningcited by 3
  5. 2026Cross4D-JEPA: Dense Cross-modal Correspondence Distillation for 4D Point Cloud Representation Learningcited by 1
  6. 2026POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shakingcited by 0