mlsanityData to 7 Oct 2026

Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation

arXiv 2204.00570 ↗First posted 1 Apr 2022cs.LG cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

We consider unsupervised domain adaptation (UDA), where labeled data from a source domain (e.g., photographs) and unlabeled data from a target domain (e.g., sketches) are used to learn a classifier for the target domain. Conventional UDA methods (e.g., domain adversarial training) learn domain-invariant features to improve generalization to the target domain. In this paper, we show that contrastive pre-training, which learns features on unlabeled source and target data and then fine-tunes on labeled source data, is competitive with strong UDA methods. However, we find that contrastive pre-training does not learn domain-invariant features, diverging from conventional UDA intuitions. We show theoretically that contrastive pre-training can learn features that vary subtantially across domains but still generalize to the target domain, by disentangling domain and class information. Our results suggest that domain invariance is not necessary for UDA. We empirically validate our theory on benchmark vision datasets.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 37 of its 64 references that are in the map. First posted on arXiv 1 Apr 2022.

Builds on

37 of 64 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  11. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  12. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
25 more
  1. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  2. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  3. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  4. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  5. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  6. 2019Contrastive Adaptation Network for Unsupervised Domain Adaptationcited by 407
  7. 2017Community Detection and Stochastic Block Modelscited by 317
  8. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  9. 2018A DIRT-T Approach to Unsupervised Domain Adaptationcited by 215
  10. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 179
  11. 2020BREEDS: Benchmarks for Subpopulation Shiftcited by 155
  12. 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
  13. 2013Consistency of spectral clustering in stochastic block modelscited by 128
  14. 2018Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttimecited by 122
  15. 2019Unsupervised Domain Adaptation through Self-Supervisioncited by 116
  16. 2021Extending the WILDS Benchmark for Unsupervised Adaptationcited by 85
  17. 2020Maximum Density Divergence for Domain Adaptationcited by 76
  18. 2021Cross-domain Contrastive Learning for Unsupervised Domain Adaptationcited by 75
  19. 2020Contrastive estimation reveals topic posterior information to linear modelscited by 54
  20. 2021Dynamic Weighted Learning for Unsupervised Domain Adaptationcited by 53
  21. 2021A Theory of Label Propagation for Subpopulation Shiftcited by 48
  22. 2019Stochastic Blockmodels meet Graph Neural Networkscited by 39
  23. 2019On Learning Invariant Representation for Domain Adaptationcited by 38
  24. 2021Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistencycited by 19
  25. 2016The Stochastic Topic Block Model for the Clustering of Vertices in Networks with Textual Edgescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 25; 2024: 10; 2025: 11; 2026: 10.

66 papers.

Sort:

By area:

  1. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  2. 2022On the duality between contrastive and non-contrastive self-supervised learningcited by 107
  3. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 85
  4. 2022Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulationscited by 44
  5. 2023InfoNCE Loss Provably Learns Cluster-Preserving Representationscited by 37
  6. 2022Contrastive Learning for Unsupervised Domain Adaptation of Time Seriescited by 36
  7. 2022RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustnesscited by 27
  8. 2023The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learningcited by 27
  9. 2023ManyDG: Many-domain Generalization for Healthcare Applicationscited by 23
  10. 2023A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learningcited by 22
  11. 2023Matrix Information Theory for Self-Supervised Learningcited by 21
  12. 2023When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysiscited by 21
38 more
  1. 2022First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domainscited by 20
  2. 2023Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regressioncited by 20
  3. 2022Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysiscited by 19
  4. 2024SF(DA)2: Source-free Domain Adaptation Through the Lens of Data Augmentationcited by 19
  5. 2023Rethinking Weak Supervision in Helping Contrastive Learningcited by 18
  6. 2023Unsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Trainingcited by 12
  7. 2022Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistencycited by 11
  8. 2024Feature contamination: Neural networks learn uncorrelated features and fail to generalizecited by 11
  9. 2023ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representationscited by 10
  10. 2025Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimensioncited by 10
  11. 2023MetaAdapt: Domain Adaptive Few-Shot Misinformation Detection via Meta Learningcited by 8
  12. 2023HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shiftscited by 8
  13. 2023Enabling Resource-efficient AIoT System with Cross-level Optimization: A surveycited by 8
  14. 2023Holistic Transfer: Towards Non-Disruptive Fine-Tuning with Partial Target Datacited by 7
  15. 2024Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentationscited by 7
  16. 2024Prototypical Partial Optimal Transport for Universal Domain Adaptationcited by 7
  17. 2024When and How Does In-Distribution Label Help Out-of-Distribution Detection?cited by 6
  18. 2023Substance or Style: What Does Your Image Embedding Know?cited by 5
  19. 2023Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shiftcited by 5
  20. 2023Harnessing small projectors and multiple views for efficient vision pretrainingcited by 5
  21. 2024How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?cited by 5
  22. 2025Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learningcited by 5
  23. 2026A Generalization Theory for JEPA-Based World Modelscited by 5
  24. 2023Unsupervised domain adaptation by learning using privileged informationcited by 3
  25. 2024Bridging OOD Detection and Generalization: A Graph-Theoretic Viewcited by 3
  26. 2025An Augmentation-Aware Theory for Self-Supervised Contrastive Learningcited by 3
  27. 2025Discovering Hierarchy-Grounded Domains with Adaptive Granularity for Clinical Domain Generalizationcited by 3
  28. 2026Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learningcited by 3
  29. 2023Contrast and Clustering: Learning Neighborhood Pair Representation for Source-free Domain Adaptationcited by 2
  30. 2024Bridging Domains with Approximately Shared Featurescited by 2
  31. 2024Views Can Be Deceiving: Improved SSL Through Feature Space Augmentationcited by 2
  32. 2025Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspectivecited by 2
  33. 2025Gradual Domain Adaptation for Graph Learningcited by 2
  34. 2025On the Alignment Between Supervised and Self-Supervised Contrastive Learningcited by 2
  35. 2022Ensemble diverse hypotheses and knowledge distillation for unsupervised cross-subject adaptationcited by 1
  36. 2023Benchmarking Low-Shot Robustness to Natural Distribution Shiftscited by 1
  37. 2023Detecting and Learning Out-of-Distribution Data in the Open world: Algorithm and Theorycited by 1
  38. 2025Improving Out-of-Domain Robustness with Targeted Augmentation in Frequency and Pixel Spacescited by 1

50 most cited of 66 All 66 (JSON)