# Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation

arXiv: https://arxiv.org/abs/1804.00644 · HTML: https://mlsanity.com/paper/1804.00644/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2018.

## Built on by (most cited first, 28 of 28)

- [Does Knowledge Distillation Really Work?](https://mlsanity.com/paper/2106.05945/) (2021)
- [Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition](https://mlsanity.com/paper/2102.01380/) (2021)
- [A Survey of Unsupervised Domain Adaptation for Visual Recognition](https://mlsanity.com/paper/2112.06745/) (2021)
- [Cross-Domain Ensemble Distillation for Domain Generalization](https://mlsanity.com/paper/2211.14058/) (2022)
- [Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition](https://mlsanity.com/paper/2106.02302/) (2021)
- [RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictability](https://mlsanity.com/paper/2309.00082/) (2023)
- [Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition](https://mlsanity.com/paper/2110.05354/) (2021)
- [Adversarial Imitation Learning with Trajectorial Augmentation and Correction](https://mlsanity.com/paper/2103.13887/) (2021)
- [PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations](https://mlsanity.com/paper/2203.16965/) (2022)
- [Deep Spherical Manifold Gaussian Kernel for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2105.02089/) (2021)
- [FedDKD: Federated Learning with Decentralized Knowledge Distillation](https://mlsanity.com/paper/2205.00706/) (2022)
- [StyleAM: Perception-Oriented Unsupervised Domain Adaption for Non-reference Image Quality Assessment](https://mlsanity.com/paper/2207.14489/) (2022)
- [Modular Domain Adaptation for Conformer-Based Streaming ASR](https://mlsanity.com/paper/2305.13408/) (2023)
- [Narrowing the Gap: Improved Detector Training with Noisy Location Annotations](https://mlsanity.com/paper/2206.05708/) (2022)
- [Unsupervised Domain Expansion for Visual Categorization](https://mlsanity.com/paper/2104.00233/) (2021)
- [Enhanced Separable Disentanglement for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.11915/) (2021)
- [Deep Least Squares Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2111.02207/) (2021)
- [Domain Generalization for Prostate Segmentation in Transrectal Ultrasound Images: A Multi-center Study](https://mlsanity.com/paper/2209.02126/) (2022)
- [Black-box Generalization of Machine Teaching](https://mlsanity.com/paper/2206.15205/) (2022)
- [Label Alignment Regularization for Distribution Shift](https://mlsanity.com/paper/2211.14960/) (2022)
- [Dataset Dictionary Learning in a Wasserstein Space for Federated Domain Adaptation](https://mlsanity.com/paper/2407.11647/) (2024)
- [Optimal Transport-based Adaptation in Dysarthric Speech Tasks](https://mlsanity.com/paper/2104.02535/) (2021)
- [UBR2S: Uncertainty-Based Resampling and Reweighting Strategy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2110.11739/) (2021)
- [Certainty Volume Prediction for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2111.02901/) (2021)
- [End to End Generative Meta Curriculum Learning For Medical Data Augmentation](https://mlsanity.com/paper/2212.10086/) (2022)
- [Deep Feature Registration for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2310.16100/) (2023)
- [Noise robust distillation of self-supervised speech models via correlation metrics](https://mlsanity.com/paper/2312.12153/) (2023)
- [Transfer Reinforcement Learning in Heterogeneous Action Spaces using Subgoal Mapping](https://mlsanity.com/paper/2410.14484/) (2024)

## Abstract (arXiv)

The teacher-student (T/S) learning has been shown effective in unsupervised domain adaptation [1]. It is a form of transfer learning, not in terms of the transfer of recognition decisions, but the knowledge of posteriori probabilities in the source domain as evaluated by the teacher model. It learns to handle the speaker and environment variability inherent in and restricted to the speech signal in the target domain without proactively addressing the robustness to other likely conditions. Performance degradation may thus ensue. In this work, we advance T/S learning by proposing adversarial T/S learning to explicitly achieve condition-robust unsupervised domain adaptation. In this method, a student acoustic model and a condition classifier are jointly optimized to minimize the Kullback-Leibler divergence between the output distributions of the teacher and student models, and simultaneously, to min-maximize the condition classification loss. A condition-invariant deep feature is learned in the adapted student model through this procedure. We further propose multi-factorial adversarial T/S learning which suppresses condition variabilities caused by multiple factors simultaneously. Evaluated with the noisy CHiME-3 test set, the proposed methods achieve relative word error rate improvements of 44.60% and 5.38%, respectively, over a clean source model and a strong T/S learning baseline model.