# SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets

arXiv: https://arxiv.org/abs/2308.11880 · HTML: https://mlsanity.com/paper/2308.11880/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 40 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [Instance Adaptive Self-Training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.12197/) (2020)
- [Source-Free Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16372/) (2021)
- [Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking](https://mlsanity.com/paper/2109.03805/) (2021)
- [NLNL: Negative Learning for Noisy Labels](https://mlsanity.com/paper/1908.07387/) (2019)
- [Self-supervised Augmentation Consistency for Adapting Semantic Segmentation](https://mlsanity.com/paper/2105.00097/) (2021)
- [Progressive Domain Adaptation for Object Detection](https://mlsanity.com/paper/1910.11319/) (2019)
- [xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/1911.12676/) (2019)
- [Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1909.00589/) (2019)
- [All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentation](https://mlsanity.com/paper/1903.12212/) (2019)
- [Unsupervised Multi-source Domain Adaptation Without Access to Source Data](https://mlsanity.com/paper/2104.01845/) (2021)
- [Domain Adaptive Semantic Segmentation Using Weak Labels](https://mlsanity.com/paper/2007.15176/) (2020)
- [Cross-domain Imitation from Observations](https://mlsanity.com/paper/2105.10037/) (2021)
- [Effective Use of Synthetic Data for Urban Scene Semantic Segmentation](https://mlsanity.com/paper/1807.06132/) (2018)
- [Learning from Scale-Invariant Examples for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2007.14449/) (2020)
- [Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training](https://mlsanity.com/paper/1810.07911/) (2018)
- [Semantically Adaptive Image-to-image Translation for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/2009.01166/) (2020)

## Built on by (most cited first, 5 of 5)

- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [Test-Time Distillation for Continual Model Adaptation](https://mlsanity.com/paper/2506.02671/) (2025)
- [SAM-guided Pseudo Label Enhancement for Multi-modal 3D Semantic Segmentation](https://mlsanity.com/paper/2502.00960/) (2025)
- [Interactive Test-Time Adaptation with Reliable Spatial-Temporal Voxels for Multi-Modal Segmentation](https://mlsanity.com/paper/2403.06461/) (2024)
- [PanDA: Unsupervised Domain Adaptation for Multimodal 3D Panoptic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2604.19379/) (2026)

## Abstract (arXiv)

Scene understanding using multi-modal data is necessary in many applications, e.g., autonomous navigation. To achieve this in a variety of situations, existing models must be able to adapt to shifting data distributions without arduous data annotation. Current approaches assume that the source data is available during adaptation and that the source consists of paired multi-modal data. Both these assumptions may be problematic for many applications. Source data may not be available due to privacy, security, or economic concerns. Assuming the existence of paired multi-modal data for training also entails significant data collection costs and fails to take advantage of widely available freely distributed pre-trained uni-modal models. In this work, we relax both of these assumptions by addressing the problem of adapting a set of models trained independently on uni-modal data to a target domain consisting of unlabeled multi-modal data, without having access to the original source dataset. Our proposed approach solves this problem through a switching framework which automatically chooses between two complementary methods of cross-modal pseudo-label fusion -- agreement filtering and entropy weighting -- based on the estimated domain gap. We demonstrate our work on the semantic segmentation problem. Experiments across seven challenging adaptation scenarios verify the efficacy of our approach, achieving results comparable to, and in some cases outperforming, methods which assume access to source data. Our method achieves an improvement in mIoU of up to 12% over competing baselines. Our code is publicly available at https://github.com/csimo005/SUMMIT.