# Strong Baselines for Neural Semi-supervised Learning under Domain Shift

arXiv: https://arxiv.org/abs/1804.09530 · HTML: https://mlsanity.com/paper/1804.09530/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2018.

## Built on by (most cited first, 30 of 39)

- [SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption](https://mlsanity.com/paper/2106.15147/) (2021)
- [Debiased Self-Training for Semi-Supervised Learning](https://mlsanity.com/paper/2202.07136/) (2022)
- [UDALM: Unsupervised Domain Adaptation through Language Modeling](https://mlsanity.com/paper/2104.07078/) (2021)
- [M2D2: A Massively Multi-domain Language Modeling Dataset](https://mlsanity.com/paper/2210.07370/) (2022)
- [Pose-disentangled Contrastive Learning for Self-supervised Facial Representation](https://mlsanity.com/paper/2211.13490/) (2022)
- [Rethinking Semi-supervised Learning with Language Models](https://mlsanity.com/paper/2305.13002/) (2023)
- [Analysing The Impact Of Linguistic Features On Cross-Lingual Transfer](https://mlsanity.com/paper/2105.05975/) (2021)
- [Unsupervised Domain Adaptation with Adapter](https://mlsanity.com/paper/2111.00667/) (2021)
- [Zero-shot Transfer of Article-aware Legal Outcome Classification for European Court of Human Rights Cases](https://mlsanity.com/paper/2302.00609/) (2023)
- [SemiReward: A General Reward Model for Semi-supervised Learning](https://mlsanity.com/paper/2310.03013/) (2023)
- [Active Learning Over Multiple Domains in Natural Language Tasks](https://mlsanity.com/paper/2202.00254/) (2022)
- [Self-Ensembling Contrastive Learning for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2105.12924/) (2021)
- [Semi-Supervised Text Classification via Self-Pretraining](https://mlsanity.com/paper/2109.15300/) (2021)
- [Exploiting Image Translations via Ensemble Self-Supervised Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2107.06235/) (2021)
- [TADA: Efficient Task-Agnostic Domain Adaptation for Transformers](https://mlsanity.com/paper/2305.12717/) (2023)
- [Domain Adaptation for Sentiment Analysis Using Increased Intraclass Separation](https://mlsanity.com/paper/2107.01598/) (2021)
- [AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Text](https://mlsanity.com/paper/2503.18247/) (2025)
- [Industry Scale Semi-Supervised Learning for Natural Language Understanding](https://mlsanity.com/paper/2103.15871/) (2021)
- [Harnessing Unlabeled Data to Improve Generalization of Biometric Gender and Age Classifiers](https://mlsanity.com/paper/2110.04427/) (2021)
- [Exploiting Unlabeled Data for Target-Oriented Opinion Words Extraction](https://mlsanity.com/paper/2208.08280/) (2022)
- [Generalizing through Forgetting -- Domain Generalization for Symptom Event Extraction in Clinical Notes](https://mlsanity.com/paper/2209.09485/) (2022)
- [Data Augmentation for Modeling Human Personality: The Dexter Machine](https://mlsanity.com/paper/2301.08606/) (2023)
- [A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets](https://mlsanity.com/paper/2304.00483/) (2023)
- [Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets](https://mlsanity.com/paper/2401.00575/) (2023)
- [New Semantic Task for the French Spoken Language Understanding MEDIA Benchmark](https://mlsanity.com/paper/2403.19727/) (2024)
- [Interactive Machine Teaching by Labeling Rules and Instances](https://mlsanity.com/paper/2409.05199/) (2024)
- [Revisiting Tri-training of Dependency Parsers](https://mlsanity.com/paper/2109.08122/) (2021)
- [Rethinking and Recomputing the Value of Machine Learning Models](https://mlsanity.com/paper/2209.15157/) (2022)
- [Optimizing Multi-Domain Performance with Active Learning-based Improvement Strategies](https://mlsanity.com/paper/2304.06277/) (2023)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)

## Abstract (arXiv)

Novel neural models have been proposed in recent years for learning under domain shift. Most models, however, only evaluate on a single task, on proprietary datasets, or compare to weak baselines, which makes comparison of models difficult. In this paper, we re-evaluate classic general-purpose bootstrapping approaches in the context of neural networks under domain shifts vs. recent neural approaches and propose a novel multi-task tri-training method that reduces the time and space complexity of classic tri-training. Extensive experiments on two benchmarks are negative: while our novel method establishes a new state-of-the-art for sentiment analysis, it does not fare consistently the best. More importantly, we arrive at the somewhat surprising conclusion that classic tri-training, with some additions, outperforms the state of the art. We conclude that classic approaches constitute an important and strong baseline.