# DaMSTF: Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation

arXiv: https://arxiv.org/abs/2308.02753 · HTML: https://mlsanity.com/paper/2308.02753/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 5 Aug 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Bilevel Programming for Hyperparameter Optimization and Meta-Learning](https://mlsanity.com/paper/1806.04910/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks](https://mlsanity.com/paper/2001.06362/) (2020)
- [Asymmetric Tri-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1702.08400/) (2017)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [Cycle Self-Training for Domain Adaptation](https://mlsanity.com/paper/2103.03571/) (2021)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [Two-phase Pseudo Label Densification for Self-training based Domain Adaptation](https://mlsanity.com/paper/2012.04828/) (2020)
- [Bilateral Adversarial Training: Towards Fast Training of More Robust Models Against Adversarial Attacks](https://mlsanity.com/paper/1811.10716/) (2018)
- [Learning Reporting Dynamics during Breaking News for Rumour Detection in Social Media](https://mlsanity.com/paper/1610.07363/) (2016)
- [Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification](https://mlsanity.com/paper/1809.00530/) (2018)
- [Meta-Learning for Neural Relation Classification with Distant Supervision](https://mlsanity.com/paper/2010.13544/) (2020)

## Built on by (most cited first, 2 of 2)

- [Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling](https://mlsanity.com/paper/2409.11283/) (2024)
- [Signs of Struggle: Spotting Cognitive Distortions across Language and Register](https://mlsanity.com/paper/2508.20771/) (2025)

## Abstract (arXiv)

Self-training emerges as an important research line on domain adaptation. By taking the model's prediction as the pseudo labels of the unlabeled data, self-training bootstraps the model with pseudo instances in the target domain. However, the prediction errors of pseudo labels (label noise) challenge the performance of self-training. To address this problem, previous approaches only use reliable pseudo instances, i.e., pseudo instances with high prediction confidence, to retrain the model. Although these strategies effectively reduce the label noise, they are prone to miss the hard examples. In this paper, we propose a new self-training framework for domain adaptation, namely Domain adversarial learning enhanced Self-Training Framework (DaMSTF). Firstly, DaMSTF involves meta-learning to estimate the importance of each pseudo instance, so as to simultaneously reduce the label noise and preserve hard examples. Secondly, we design a meta constructor for constructing the meta-validation set, which guarantees the effectiveness of the meta-learning module by improving the quality of the meta-validation set. Thirdly, we find that the meta-learning module suffers from the training guidance vanishment and tends to converge to an inferior optimal. To this end, we employ domain adversarial learning as a heuristic neural network initialization method, which can help the meta-learning module converge to a better optimal. Theoretically and experimentally, we demonstrate the effectiveness of the proposed DaMSTF. On the cross-domain sentiment classification task, DaMSTF improves the performance of BERT with an average of nearly 4%.