mlsanityData to 7 Oct 2026

Domain Confused Contrastive Learning for Unsupervised Domain Adaptation

arXiv 2207.04564 ↗First posted 10 Jul 2022cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In this work, we study Unsupervised Domain Adaptation (UDA) in a challenging self-supervised approach. One of the difficulties is how to learn task discrimination in the absence of target labels. Unlike previous literature which directly aligns cross-domain distributions or leverages reverse gradient, we propose Domain Confused Contrastive Learning (DCCL) to bridge the source and the target domains via domain puzzles, and retain discriminative representations after adaptation. Technically, DCCL searches for a most domain-challenging direction and exquisitely crafts domain confused augmentations as positive pairs, then it contrastively encourages the model to pull representations towards the other domain, thus learning more stable and effective domain invariances. We also investigate whether contrastive learning necessarily helps with UDA when performing other data augmentations. Extensive experiments demonstrate that DCCL significantly outperforms baselines.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 10 Jul 2022.

Builds on

44 of 63 references.

References · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  10. 2020Supervised Contrastive Learningcited by 2,803
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
32 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  5. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  6. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  7. 2015Image-based Recommendations on Styles and Substitutescited by 702
  8. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  9. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  10. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  11. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  12. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  13. 2018Characterizing and Avoiding Negative Transfercited by 226
  14. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  15. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  16. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  17. 2017Asymmetric Tri-training for Unsupervised Domain Adaptationcited by 152
  18. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 142
  19. 2021Cross-domain Contrastive Learning for Unsupervised Domain Adaptationcited by 75
  20. 2019Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptationcited by 62
  21. 2019Domain Adaptation with Asymmetrically-Relaxed Distribution Alignmentcited by 61
  22. 2019"Mask and Infill" : Applying Masked Language Model to Sentiment Transfercited by 48
  23. 2020CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understandingcited by 41
  24. 2020PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Modelscited by 33
  25. 2021UDALM: Unsupervised Domain Adaptation through Language Modelingcited by 33
  26. 2018Adversarial Domain Adaptation for Duplicate Question Detectioncited by 30
  27. 2020Multi-Source Domain Adaptation for Text Classification via DistanceNet-Banditscited by 30
  28. 2020Joint Contrastive Learning for Unsupervised Domain Adaptationcited by 25
  29. 2016Neural Structural Correspondence Learning for Domain Adaptationcited by 19
  30. 2018Adaptive Semi-supervised Learning for Cross-domain Sentiment Classificationcited by 16
  31. 2019Simplified Neural Unsupervised Domain Adaptationcited by 10
  32. 2021Targeted Adversarial Training for Natural Language Understandingcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 1.

4 papers.

  1. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  2. 2022Improving the Sample Efficiency of Prompt Tuning with Domain Adaptationcited by 15
  3. 2024How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?cited by 5
  4. 2022Understanding Domain Learning in Language Models Through Subpopulation Analysiscited by 4