# Joint Learning of Pre-Trained and Random Units for Domain Adaptation in Part-of-Speech Tagging

arXiv: https://arxiv.org/abs/1904.03595 · HTML: https://mlsanity.com/paper/1904.03595/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2019.

## Built on by (most cited first, 2 of 2)

- [Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units](https://mlsanity.com/paper/2106.04935/) (2021)
- [Unsupervised Domain Adaptation using Lexical Transformations and Label Injection for Twitter Data](https://mlsanity.com/paper/2307.10210/) (2023)

## Abstract (arXiv)

Fine-tuning neural networks is widely used to transfer valuable knowledge from high-resource to low-resource domains. In a standard fine-tuning scheme, source and target problems are trained using the same architecture. Although capable of adapting to new domains, pre-trained units struggle with learning uncommon target-specific patterns. In this paper, we propose to augment the target-network with normalised, weighted and randomly initialised units that beget a better adaptation while maintaining the valuable source knowledge. Our experiments on POS tagging of social media texts (Tweets domain) demonstrate that our method achieves state-of-the-art performances on 3 commonly used datasets.