# Self-supervised Modal and View Invariant Feature Learning

arXiv: https://arxiv.org/abs/2005.14169 · HTML: https://mlsanity.com/paper/2005.14169/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2020.

## Built on by (most cited first, 20 of 20)

- [Self-Supervised Pretraining of 3D Features on any Point-Cloud](https://mlsanity.com/paper/2101.02691/) (2021)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes](https://mlsanity.com/paper/2111.13260/) (2021)
- [PointGPT: Auto-regressively Generative Pre-training from Point Clouds](https://mlsanity.com/paper/2305.11487/) (2023)
- [Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness](https://mlsanity.com/paper/2504.01901/) (2025)
- [Learning 3D Semantic Segmentation with only 2D Image Supervision](https://mlsanity.com/paper/2110.11325/) (2021)
- [Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data](https://mlsanity.com/paper/2206.02353/) (2022)
- [Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Models](https://mlsanity.com/paper/2305.08776/) (2023)
- [Spatiotemporal Self-supervised Learning for Point Clouds in the Wild](https://mlsanity.com/paper/2303.16235/) (2023)
- [Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images](https://mlsanity.com/paper/2306.07598/) (2023)
- [Self-Supervised Learning with Multi-View Rendering for 3D Point Cloud Analysis](https://mlsanity.com/paper/2210.15904/) (2022)
- [Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data](https://mlsanity.com/paper/2407.10200/) (2024)
- [A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds](https://mlsanity.com/paper/2104.05164/) (2021)
- [Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange](https://mlsanity.com/paper/2404.07504/) (2024)
- [SeRP: Self-Supervised Representation Learning Using Perturbed Point Clouds](https://mlsanity.com/paper/2209.06067/) (2022)
- [SAM-Guided Masked Token Prediction for 3D Scene Understanding](https://mlsanity.com/paper/2410.12158/) (2024)
- [Point Cloud Self-supervised Learning via 3D to Multi-view Masked Learner](https://mlsanity.com/paper/2311.10887/) (2023)
- [Mitigating Prior Shape Bias in Point Clouds via Differentiable Center Learning](https://mlsanity.com/paper/2402.02088/) (2024)
- [Point Cloud Understanding via Attention-Driven Contrastive Learning](https://mlsanity.com/paper/2411.14744/) (2024)
- [Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework](https://mlsanity.com/paper/2605.17566/) (2026)

## Abstract (arXiv)

Most of the existing self-supervised feature learning methods for 3D data either learn 3D features from point cloud data or from multi-view images. By exploring the inherent multi-modality attributes of 3D objects, in this paper, we propose to jointly learn modal-invariant and view-invariant features from different modalities including image, point cloud, and mesh with heterogeneous networks for 3D data. In order to learn modal- and view-invariant features, we propose two types of constraints: cross-modal invariance constraint and cross-view invariant constraint. Cross-modal invariance constraint forces the network to maximum the agreement of features from different modalities for same objects, while the cross-view invariance constraint forces the network to maximum agreement of features from different views of images for same objects. The quality of learned features has been tested on different downstream tasks with three modalities of data including point cloud, multi-view images, and mesh. Furthermore, the invariance cross different modalities and views are evaluated with the cross-modal retrieval task. Extensive evaluation results demonstrate that the learned features are robust and have strong generalizability across different tasks.