# Memory-Efficient Pseudo-Labeling for Online Source-Free Universal Domain Adaptation using a Gaussian Mixture Model

arXiv: https://arxiv.org/abs/2407.14208 · HTML: https://mlsanity.com/paper/2407.14208/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Universal Domain Adaptation through Self Supervision](https://mlsanity.com/paper/2002.07953/) (2020)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Open Set Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1804.10427/) (2018)
- [Partial Adversarial Domain Adaptation](https://mlsanity.com/paper/1808.04205/) (2018)
- [Robust Mean Teacher for Continual and Gradual Test-Time Adaptation](https://mlsanity.com/paper/2211.13081/) (2022)
- [Test-Time Adaptation via Conjugate Pseudo-labels](https://mlsanity.com/paper/2207.09640/) (2022)
- [OVANet: One-vs-All Network for Universal Domain Adaptation](https://mlsanity.com/paper/2104.03344/) (2021)
- [Importance Weighted Adversarial Nets for Partial Domain Adaptation](https://mlsanity.com/paper/1803.09210/) (2018)
- [Partial Transfer Learning with Selective Adversarial Networks](https://mlsanity.com/paper/1707.07901/) (2017)
- [On the Effectiveness of Image Rotation for Open Set Domain Adaptation](https://mlsanity.com/paper/2007.12360/) (2020)
- [Towards Inheritable Models for Open-Set Domain Adaptation](https://mlsanity.com/paper/2004.04388/) (2020)
- [On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion](https://mlsanity.com/paper/2308.09942/) (2023)
- [Open Set Domain Adaptation for Image and Action Recognition](https://mlsanity.com/paper/1907.12865/) (2019)
- [Upcycling Models under Domain and Category Shift](https://mlsanity.com/paper/2303.07110/) (2023)
- [LEAD: Learning Decomposition for Source-free Universal Domain Adaptation](https://mlsanity.com/paper/2403.03421/) (2024)
- [UMAD: Universal Model Adaptation under Domain and Category Shift](https://mlsanity.com/paper/2112.08553/) (2021)

## Built on by (most cited first, 4 of 4)

- [ViLAaD: Enhancing "Attracting and Dispersing'' Source-Free Domain Adaptation with Vision-and-Language Model](https://mlsanity.com/paper/2503.23529/) (2025)
- [Analysis of Pseudo-Labeling for Online Source-Free Universal Domain Adaptation](https://mlsanity.com/paper/2504.11992/) (2025)
- [SloMo-Fast: Slow-Momentum and Fast-Adaptive Teachers for Source-Free Continual Test-Time Adaptation](https://mlsanity.com/paper/2511.18468/) (2025)
- [Autonomous Source Knowledge Selection in Multi-Domain Adaptation](https://mlsanity.com/paper/2512.14710/) (2025)

## Abstract (arXiv)

In practice, domain shifts are likely to occur between training and test data, necessitating domain adaptation (DA) to adjust the pre-trained source model to the target domain. Recently, universal domain adaptation (UniDA) has gained attention for addressing the possibility of an additional category (label) shift between the source and target domain. This means new classes can appear in the target data, some source classes may no longer be present, or both at the same time. For practical applicability, UniDA methods must handle both source-free and online scenarios, enabling adaptation without access to the source data and performing batch-wise updates in parallel with prediction. In an online setting, preserving knowledge across batches is crucial. However, existing methods often require substantial memory, which is impractical because memory is limited and valuable, in particular on embedded systems. Therefore, we consider memory-efficiency as an additional constraint. To achieve memory-efficient online source-free universal domain adaptation (SF-UniDA), we propose a novel method that continuously captures the distribution of known classes in the feature space using a Gaussian mixture model (GMM). This approach, combined with entropy-based out-of-distribution detection, allows for the generation of reliable pseudo-labels. Finally, we combine a contrastive loss with a KL divergence loss to perform the adaptation. Our approach not only achieves state-of-the-art results in all experiments on the DomainNet and Office-Home datasets but also significantly outperforms the existing methods on the challenging VisDA-C dataset, setting a new benchmark for online SF-UniDA. Our code is available at https://github.com/pascalschlachter/GMM.