# xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation

arXiv: https://arxiv.org/abs/1911.12676 · HTML: https://mlsanity.com/paper/1911.12676/

## Key facts

- Cited by 129 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2019.

## Built on by (most cited first, 30 of 129)

- [ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection](https://mlsanity.com/paper/2103.05346/) (2021)
- [2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds](https://mlsanity.com/paper/2207.04397/) (2022)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [Robo3D: Towards Robust and Reliable 3D Perception against Corruptions](https://mlsanity.com/paper/2303.17597/) (2023)
- [Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Data](https://mlsanity.com/paper/2203.16258/) (2022)
- [Segment Any Point Cloud Sequences by Distilling Vision Foundation Models](https://mlsanity.com/paper/2306.09347/) (2023)
- [Bidirectional Projection Network for Cross Dimension Scene Understanding](https://mlsanity.com/paper/2103.14326/) (2021)
- [MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.12667/) (2022)
- [PolarMix: A General Data Augmentation Technique for LiDAR Point Clouds](https://mlsanity.com/paper/2208.00223/) (2022)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency](https://mlsanity.com/paper/2107.11355/) (2021)
- [Learning 3D Semantic Segmentation with only 2D Image Supervision](https://mlsanity.com/paper/2110.11325/) (2021)
- [EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation](https://mlsanity.com/paper/2106.15277/) (2021)
- [ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection](https://mlsanity.com/paper/2108.06682/) (2021)
- [LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection](https://mlsanity.com/paper/2203.14956/) (2022)
- [CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentation](https://mlsanity.com/paper/2207.09778/) (2022)
- [Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2405.05258/) (2024)
- [A Survey on Deep Domain Adaptation for LiDAR Perception](https://mlsanity.com/paper/2106.02377/) (2021)
- [Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2107.14724/) (2021)
- [Three Pillars improving Vision Foundation Model Distillation for Lidar](https://mlsanity.com/paper/2310.17504/) (2023)
- [Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval](https://mlsanity.com/paper/2209.11572/) (2022)
- [Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers](https://mlsanity.com/paper/2307.04129/) (2023)
- [ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation](https://mlsanity.com/paper/2111.15242/) (2021)
- [Cross-modal Learning for Domain Adaptation in 3D Semantic Segmentation](https://mlsanity.com/paper/2101.07253/) (2021)
- [GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation](https://mlsanity.com/paper/2207.09763/) (2022)
- [Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2303.11203/) (2023)
- [4D Contrastive Superflows are Dense 3D Representation Learners](https://mlsanity.com/paper/2407.06190/) (2024)
- [TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation](https://mlsanity.com/paper/2303.16730/) (2023)
- [Deformation and Correspondence Aware Unsupervised Synthetic-to-Real Scene Flow Estimation for Point Clouds](https://mlsanity.com/paper/2203.16895/) (2022)
- [MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities](https://mlsanity.com/paper/2405.17419/) (2024)

## Abstract (arXiv)

Unsupervised Domain Adaptation (UDA) is crucial to tackle the lack of annotations in a new domain. There are many multi-modal datasets, but most UDA approaches are uni-modal. In this work, we explore how to learn from multi-modality and propose cross-modal UDA (xMUDA) where we assume the presence of 2D images and 3D point clouds for 3D semantic segmentation. This is challenging as the two input spaces are heterogeneous and can be impacted differently by domain shift. In xMUDA, modalities learn from each other through mutual mimicking, disentangled from the segmentation objective, to prevent the stronger modality from adopting false predictions from the weaker one. We evaluate on new UDA scenarios including day-to-night, country-to-country and dataset-to-dataset, leveraging recent autonomous driving datasets. xMUDA brings large improvements over uni-modal UDA on all tested scenarios, and is complementary to state-of-the-art UDA techniques. Code is available at https://github.com/valeoai/xmuda.