# Unsupervised Domain Adaptation using Lexical Transformations and Label Injection for Twitter Data

arXiv: https://arxiv.org/abs/2307.10210 · HTML: https://mlsanity.com/paper/2307.10210/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://mlsanity.com/paper/2004.02105/) (2020)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1806.00258/) (2018)
- [Strong Baselines for Neural Semi-supervised Learning under Domain Shift](https://mlsanity.com/paper/1804.09530/) (2018)
- [Parsing Tweets into Universal Dependencies](https://mlsanity.com/paper/1804.08228/) (2018)
- [On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation](https://mlsanity.com/paper/2109.07591/) (2021)
- [Unsupervised Self-Training for Sentiment Analysis of Code-Switched Data](https://mlsanity.com/paper/2103.14797/) (2021)
- [Domain adaptation for part-of-speech tagging of noisy user-generated text](https://mlsanity.com/paper/1905.08920/) (2019)
- [Joint Learning of Pre-Trained and Random Units for Domain Adaptation in Part-of-Speech Tagging](https://mlsanity.com/paper/1904.03595/) (2019)

## Abstract (arXiv)

Domain adaptation is an important and widely studied problem in natural language processing. A large body of literature tries to solve this problem by adapting models trained on the source domain to the target domain. In this paper, we instead solve this problem from a dataset perspective. We modify the source domain dataset with simple lexical transformations to reduce the domain shift between the source dataset distribution and the target dataset distribution. We find that models trained on the transformed source domain dataset performs significantly better than zero-shot models. Using our proposed transformations to convert standard English to tweets, we reach an unsupervised part-of-speech (POS) tagging accuracy of 92.14% (from 81.54% zero shot accuracy), which is only slightly below the supervised performance of 94.45%. We also use our proposed transformations to synthetically generate tweets and augment the Twitter dataset to achieve state-of-the-art performance for POS tagging.