Pick up the PACE: Fast and Simple Domain Adaptation via Ensemble Pseudo-Labeling

Domain Adaptation (DA) has received widespread attention from deep learning researchers in recent years because of its potential to improve test accuracy with out-of-distribution labeled data. Most state-of-the-art DA algorithms require an extensive amount of hyperparameter tuning and are computationally intensive due to the large batch sizes required. In this work, we propose a fast and simple DA method consisting of three stages: (1) domain alignment by covariance matching, (2) pseudo-labeling, and (3) ensembling. We call this method PACE, for Pseudo-labels, Alignment of Covariances, and Ensembles. PACE is trained on top of fixed features extracted from an ensemble of modern pretrained backbones. PACE exceeds previous state-of-the-art by 5 - 10 % on most benchmark adaptation tasks without training a neural network. PACE reduces training time and hyperparameter tuning time by 82% and 97%, respectively, when compared to state-of-the-art DA methods. Code is released here: https://github.com/Chris210634/PACE-Domain-Adaptation