mlsanityData to 7 Oct 2026

Mx2M: Masked Cross-Modality Modeling in Domain Adaptation for 3D Semantic Segmentation

arXiv 2307.04231 ↗First posted 9 Jul 2023cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Existing methods of cross-modal domain adaptation for 3D semantic segmentation predict results only via 2D-3D complementarity that is obtained by cross-modal feature matching. However, as lacking supervision in the target domain, the complementarity is not always reliable. The results are not ideal when the domain gap is large. To solve the problem of lacking supervision, we introduce masked modeling into this task and propose a method Mx2M, which utilizes masked cross-modality modeling to reduce the large domain gap. Our Mx2M contains two components. One is the core solution, cross-modal removal and prediction (xMRP), which makes the Mx2M adapt to various scenarios and provides cross-modal self-supervision. The other is a new way of cross-modal feature matching, the dynamic cross-modal filter (DxMF) that ensures the whole method dynamically uses more suitable 2D-3D complementarity. Evaluation of the Mx2M on three DA scenarios, including Day/Night, USA/Singapore, and A2D2/SemanticKITTI, brings large improvements over previous methods on many metrics.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 9 Jul 2023.

Builds on

37 of 50 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  12. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
25 more
  1. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  2. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  3. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  4. 2018Learning to Adapt Structured Output Space for Semantic Segmentationcited by 584
  5. 2019Bidirectional Learning for Domain Adaptation of Semantic Segmentationcited by 308
  6. 2018Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptationcited by 295
  7. 2019SOLO: Segmenting Objects by Locationscited by 289
  8. 2016FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptationcited by 242
  9. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  10. 2021Pretrained Transformers as Universal Computation Enginescited by 162
  11. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  12. 20183DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentationcited by 118
  13. 2019PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representationcited by 114
  14. 2021VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistencycited by 114
  15. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95
  16. 2020Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistencycited by 77
  17. 2017Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptationcited by 70
  18. 2021Learning 3D Semantic Segmentation with only 2D Image Supervisioncited by 60
  19. 20183D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning from a 2D Trained Networkcited by 47
  20. 20213D-to-2D Distillation for Indoor Scene Parsingcited by 47
  21. 2021Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentationcited by 46
  22. 2021Revisiting the Transferability of Supervised Pretraining: an MLP Perspectivecited by 36
  23. 2021Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separationcited by 27
  24. 2020Self-supervised Feature Learning by Cross-modality and Cross-view Correspondencescited by 19
  25. 2022Masked Autoencoders are Robust Data Augmentorscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 2.

10 papers.

  1. 2024Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalitiescited by 29
  2. 2024Masked Image Modeling: A Surveycited by 28
  3. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  4. 2024Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentationcited by 21
  5. 2024MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selectioncited by 12
  6. 2025Reducing Unimodal Bias in Multi-Modal Semantic Segmentation with Multi-Scale Functional Entropy Regularizationcited by 9
  7. 2025Improving Multimodal Distillation for 3D Semantic Segmentation under Domain Shiftcited by 2
  8. 20252D-3D Attention and Entropy for Pose Robust 2D Facial Recognitioncited by 1
  9. 2026PanDA: Unsupervised Domain Adaptation for Multimodal 3D Panoptic Segmentation in Autonomous Drivingcited by 0
  10. 2026UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decompositioncited by 0