mlsanityData to 14 Oct 2026

Exploiting the Complementarity of 2D and 3D Networks to Address Domain-Shift in 3D Semantic Segmentation

arXiv 2304.02991 ↗First posted 6 Apr 2023cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

3D semantic segmentation is a critical task in many real-world applications, such as autonomous driving, robotics, and mixed reality. However, the task is extremely challenging due to ambiguities coming from the unstructured, sparse, and uncolored nature of the 3D point clouds. A possible solution is to combine the 3D information with others coming from sensors featuring a different modality, such as RGB cameras. Recent multi-modal 3D semantic segmentation networks exploit these modalities relying on two branches that process the 2D and 3D information independently, striving to maintain the strength of each modality. In this work, we first explain why this design choice is effective and then show how it can be improved to make the multi-modal semantic segmentation more robust to domain shift. Our surprisingly simple contribution achieves state-of-the-art performances on four popular multi-modal unsupervised domain adaptation benchmarks, as well as better results in a domain generalization scenario.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 62 of its 78 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

62 of 78 references.

References · 62

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  6. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  7. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
  8. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  9. 2018Deep Visual Domain Adaptation: A Surveycited by 635
  10. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  11. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  12. 2018Learning to Adapt Structured Output Space for Semantic Segmentationcited by 584
38 more
  1. 2019Deep Learning for 3D Point Clouds: A Surveycited by 558
  2. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 553
  3. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  4. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  5. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 342
  6. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 341
  7. 2019Bidirectional Learning for Domain Adaptation of Semantic Segmentationcited by 308
  8. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 300
  9. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
  10. 2021Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentationcited by 268
  11. 2023Multimodal Deep Learningcited by 258
  12. 2016FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptationcited by 243
  13. 2018SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloudcited by 228
  14. 2020A2D2: Audi Autonomous Driving Datasetcited by 190
  15. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  16. 2017Image to Image Translation for Domain Adaptationcited by 171
  17. 2022HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentationcited by 164
  18. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 158
  19. 2020Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentationcited by 153
  20. 2019Domain Adaptation for Structured Output via Discriminative Patch Representationscited by 150
  21. 2020Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentationcited by 142
  22. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  23. 2019Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentationcited by 126
  24. 2020Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentationcited by 126
  25. 2018Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approachcited by 119
  26. 2020Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Cloudscited by 116
  27. 2019All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentationcited by 104
  28. 2018DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptationcited by 101
  29. 2019DADA: Depth-aware Domain Adaptation in Semantic Segmentationcited by 97
  30. 2022MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentationcited by 76
  31. 2018Fully Convolutional Adaptation Networks for Semantic Segmentationcited by 75
  32. 2018Self-Supervised Model Adaptation for Multimodal Semantic Segmentationcited by 73
  33. 2017Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptationcited by 71
  34. 2020ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework for LiDAR Point Cloud Segmentationcited by 56
  35. 2022CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentationcited by 54
  36. 2019An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentationcited by 48
  37. 2021Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentationcited by 46
  38. 2021Cross-modal Learning for Domain Adaptation in 3D Semantic Segmentationcited by 40

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 3; 2025: 4; 2026: 3.

14 papers.

Sort:

By area:

  1. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  2. 2023Learning to Adapt SAM for Segmenting Cross-domain Point Cloudscited by 20
  3. 2024Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 15
  4. 2023MOHO: Learning Single-view Hand-held Object Reconstruction with Multi-view Occlusion-Aware Supervisioncited by 12
  5. 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12
  6. 20243D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidancecited by 12
  7. 2024Fusion-then-Distillation: Toward Cross-modal Positive Distillation for Domain Adaptive 3D Semantic Segmentationcited by 4
  8. 2023Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-trainingcited by 3
  9. 2025Improving Multimodal Distillation for 3D Semantic Segmentation under Domain Shiftcited by 2
  10. 2025Exploring Modality Guidance to Enhance VFM-based Feature Fusion for UDA in 3D Semantic Segmentationcited by 1
  11. 20253D Weakly Supervised Semantic Segmentation via Class-Aware and Geometry-Guided Pseudo-Label Refinementcited by 1
  12. 2026TOLiD: Bridging the Architecture Gap in Vision Foundation Model to LiDAR Pretraining via Token Lifting for Distillationcited by 1
2 more
  1. 2026PanDA: Unsupervised Domain Adaptation for Multimodal 3D Panoptic Segmentation in Autonomous Drivingcited by 0
  2. 2026UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decompositioncited by 0