# Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data

arXiv: https://arxiv.org/abs/2110.03374 · HTML: https://mlsanity.com/paper/2110.03374/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- Builds on 84 of its 117 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 30 of 84)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)

## Built on by (most cited first, 30 of 152)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation](https://mlsanity.com/paper/2302.05155/) (2023)
- [Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation](https://mlsanity.com/paper/2205.04183/) (2022)
- [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022)
- [Balancing Discriminability and Transferability for Source-Free Domain Adaptation](https://mlsanity.com/paper/2206.08009/) (2022)
- [Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection](https://mlsanity.com/paper/2203.15793/) (2022)
- [Source-Free Domain Adaptation with Frozen Multimodal Foundation Model](https://mlsanity.com/paper/2311.16510/) (2023)
- [Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution](https://mlsanity.com/paper/2305.07498/) (2023)
- [C-SFDA: A Curriculum Learning Aided Self-Training Framework for Efficient Source Free Domain Adaptation](https://mlsanity.com/paper/2303.17132/) (2023)
- [3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds](https://mlsanity.com/paper/2304.00690/) (2023)
- [CoCo: A Coupled Contrastive Framework for Unsupervised Domain Adaptive Graph Classification](https://mlsanity.com/paper/2306.04979/) (2023)
- [Towards Online Domain Adaptive Object Detection](https://mlsanity.com/paper/2204.05289/) (2022)
- [Contrastive Learning for Unsupervised Domain Adaptation of Time Series](https://mlsanity.com/paper/2206.06243/) (2022)
- [ProxyMix: Proxy-based Mixup Training with Label Refinery for Source-Free Domain Adaptation](https://mlsanity.com/paper/2205.14566/) (2022)
- [Semantics, Distortion, and Style Matter: Towards Source-free UDA for Panoramic Segmentation](https://mlsanity.com/paper/2403.12505/) (2024)
- [Unbiased Subclass Regularization for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.10026/) (2022)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [Prompting Diffusion Representations for Cross-Domain Semantic Segmentation](https://mlsanity.com/paper/2307.02138/) (2023)
- [360SFUDA++: Towards Source-free UDA for Panoramic Segmentation by Learning Reliable Category Prototypes](https://mlsanity.com/paper/2404.16501/) (2024)
- [Adversarial Alignment for Source Free Object Detection](https://mlsanity.com/paper/2301.04265/) (2023)
- [Periodically Exchange Teacher-Student for Source-Free Object Detection](https://mlsanity.com/paper/2311.13930/) (2023)
- [Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation](https://mlsanity.com/paper/2303.00914/) (2023)
- [Historical Test-time Prompt Tuning for Vision Foundation Models](https://mlsanity.com/paper/2410.20346/) (2024)
- [CAFA: Class-Aware Feature Alignment for Test-Time Adaptation](https://mlsanity.com/paper/2206.00205/) (2022)
- [Concurrent Subsidiary Supervision for Unsupervised Source-Free Domain Adaptation](https://mlsanity.com/paper/2207.13247/) (2022)
- [FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation](https://mlsanity.com/paper/2304.13672/) (2023)
- [Proxy Denoising for Source-Free Domain Adaptation](https://mlsanity.com/paper/2406.01658/) (2024)
- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [Prior Knowledge Guided Unsupervised Domain Adaptation](https://mlsanity.com/paper/2207.08877/) (2022)

## Abstract (arXiv)

Unsupervised domain adaptation aims to align a labeled source domain and an unlabeled target domain, but it requires to access the source data which often raises concerns in data privacy, data portability and data transmission efficiency. We study unsupervised model adaptation (UMA), or called Unsupervised Domain Adaptation without Source Data, an alternative setting that aims to adapt source-trained models towards target distributions without accessing source data. To this end, we design an innovative historical contrastive learning (HCL) technique that exploits historical source hypothesis to make up for the absence of source data in UMA. HCL addresses the UMA challenge from two perspectives. First, it introduces historical contrastive instance discrimination (HCID) that learns from target samples by contrasting their embeddings which are generated by the currently adapted model and the historical models. With the historical models, HCID encourages UMA to learn instance-discriminative target representations while preserving the source hypothesis. Second, it introduces historical contrastive category discrimination (HCCD) that pseudo-labels target samples to learn category-discriminative target representations. Specifically, HCCD re-weights pseudo labels according to their prediction consistency across the current and historical models. Extensive experiments show that HCL outperforms and state-of-the-art methods consistently across a variety of visual tasks and setups.