mlsanityData to 14 Oct 2026

Semi-supervised Domain Adaptation via Sample-to-Sample Self-Distillation

arXiv 2111.14353 ↗First posted 29 Nov 2021cs.CVcited by 23

On the map: Knowledge distillation → General knowledge distillation methods

Abstract (arXiv)

Semi-supervised domain adaptation (SSDA) is to adapt a learner to a new domain with only a small set of labeled samples when a large labeled dataset is given on a source domain. In this paper, we propose a pair-based SSDA method that adapts a model to the target domain using self-distillation with sample pairs. Each sample pair is composed of a teacher sample from a labeled dataset (i.e., source or labeled target) and its student sample from an unlabeled dataset (i.e., unlabeled target). Our method generates an assistant feature by transferring an intermediate style between the teacher and the student, and then train the model by minimizing the output discrepancy between the student and the assistant. During training, the assistants gradually bridge the discrepancy between the two domains, thus allowing the student to easily learn from the teacher. Experimental evaluation on standard benchmarks shows that our method effectively minimizes both the inter-domain and intra-domain discrepancies, thus achieving significant improvements over recent methods.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation. Builds on 30 of its 47 references that are in the map. First posted on arXiv 29 Nov 2021.

Builds on

30 of 47 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  5. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  6. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  8. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  9. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  10. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
  11. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  12. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
18 more
  1. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  2. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 971
  3. 2019Relational Knowledge Distillationcited by 889
  4. 2019A Closer Look at Few-shot Classificationcited by 684
  5. 2021Domain Generalization with MixStylecited by 566
  6. 2018Born Again Neural Networkscited by 546
  7. 2019Semi-supervised Domain Adaptation via Minimax Entropycited by 335
  8. 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
  9. 2019Domain-Specific Batch Normalization for Unsupervised Domain Adaptationcited by 197
  10. 2018DLOW: Domain Flow for Adaptation and Generalizationcited by 177
  11. 2019Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentationcited by 151
  12. 2020Regularizing Class-wise Predictions via Self-knowledge Distillationcited by 144
  13. 2017Adversarial Dropout Regularizationcited by 106
  14. 2019Cluster Alignment with a Teacher for Unsupervised Domain Adaptationcited by 88
  15. 2020Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptationcited by 69
  16. 2020Explaining Knowledge Distillation by Quantifying the Knowledgecited by 66
  17. 2020Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptationcited by 64
  18. 2019Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptationcited by 62

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 7; 2025: 5; 2026: 1.

23 papers.

Sort:

  1. 2024Contrastive Mean-Shift Learning for Generalized Category Discoverycited by 39
  2. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  3. 2022ConfMix: Unsupervised Domain Adaptation for Object Detection via Confidence-based Mixingcited by 28
  4. 2023A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Applicationcited by 21
  5. 2024kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabulariescited by 20
  6. 2022Combating Label Distribution Shift for Active Domain Adaptationcited by 14
  7. 2024Learning CNN on ViT: A Hybrid Model to Explicitly Class-specific Boundaries for Domain Adaptationcited by 12
  8. 2023Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge Transfercited by 10
  9. 2024Adaptive Betweenness Clustering for Semi-Supervised Domain Adaptationcited by 9
  10. 2024Enhancing Size Generalization in Graph Neural Networks through Disentangled Representation Learningcited by 7
  11. 2024Universal Semi-Supervised Domain Adaptation by Mitigating Common-Class Biascited by 3
  12. 2023Exploring Few-Shot Adaptation for Activity Recognition on Diverse Domainscited by 2
11 more
  1. 2025Weakly-Supervised Domain Adaptation with Proportion-Constrained Pseudo-Labelingcited by 2
  2. 2025Decoupling Continual Semantic Segmentationcited by 2
  3. 2022Pick up the PACE: Fast and Simple Domain Adaptation via Ensemble Pseudo-Labelingcited by 0
  4. 2022ArcAid: Analysis of Archaeological Artifacts using Drawingscited by 0
  5. 2023CAusal and collaborative proxy-tasKs lEarning for Semi-Supervised Domain Adaptationcited by 0
  6. 2023Enhancing Visual Domain Adaptation with Source Preparationcited by 0
  7. 2023GeT: Generative Target Structure Debiasing for Domain Adaptationcited by 0
  8. 2025Component Adaptive Clustering for Generalized Category Discoverycited by 0
  9. 2025MoSSDA: A Semi-Supervised Domain Adaptation Framework for Multivariate Time-Series Classification using Momentum Encodercited by 0
  10. 2025Domain Adaptation for Ulcerative Colitis Severity Estimation Using Patient-Level Diagnosescited by 0
  11. 2026A Multi-View Consistency Framework with Semi-Supervised Domain Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Federated Semi-Supervised Domain Adaptation via Knowledge Transfer
  2. 2021Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation
  3. 2025Toward Robust Semi-supervised Regression via Dual-stream Knowledge Distillation
  4. 2026Self-Specialized Teachers for Domain Post-Training
  5. 2023Unknown Sample Discovery for Source Free Open Set Domain Adaptation
  6. 2026Decoupled Optimization for Teacher-Student Semi-Supervised Learning via a Pioneer Student
  7. 2023Source-free Video Domain Adaptation by Learning from Noisy Labels
  8. 2026Adaptive Dual-Teacher Distillation with Subnetwork Rectification for Bridging Semantic Gaps in Black-Box Domain Adaptation