# Domain adaptation for part-of-speech tagging of noisy user-generated text

arXiv: https://arxiv.org/abs/1905.08920 · HTML: https://mlsanity.com/paper/1905.08920/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2019.

## Built on by (most cited first, 4 of 4)

- [Task-adaptive Pre-training of Language Models with Word Embedding Regularization](https://mlsanity.com/paper/2109.08354/) (2021)
- [Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units](https://mlsanity.com/paper/2106.04935/) (2021)
- [Active Learning for Argument Mining: A Practical Approach](https://mlsanity.com/paper/2109.13611/) (2021)
- [Unsupervised Domain Adaptation using Lexical Transformations and Label Injection for Twitter Data](https://mlsanity.com/paper/2307.10210/) (2023)

## Abstract (arXiv)

The performance of a Part-of-speech (POS) tagger is highly dependent on the domain ofthe processed text, and for many domains there is no or only very little training data available. This work addresses the problem of POS tagging noisy user-generated text using a neural network. We propose an architecture that trains an out-of-domain model on a large newswire corpus, and transfers those weights by using them as a prior for a model trained on the target domain (a data-set of German Tweets) for which there is very little an-notations available. The neural network has two standard bidirectional LSTMs at its core. However, we find it crucial to also encode a set of task-specific features, and to obtain reliable (source-domain and target-domain) word representations. Experiments with different regularization techniques such as early stopping, dropout and fine-tuning the domain adaptation prior weights are conducted. Our best model uses external weights from the out-of-domain model, as well as feature embeddings, pre-trained word and sub-word embeddings and achieves a tagging accuracy of slightly over 90%, improving on the previous state of the art for this task.