# Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment

arXiv: https://arxiv.org/abs/1910.06241 · HTML: https://mlsanity.com/paper/1910.06241/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2019.

## Built on by (most cited first, 3 of 3)

- [Fine-Tuning Transformers: Vocabulary Transfer](https://mlsanity.com/paper/2112.14569/) (2021)
- [Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks](https://mlsanity.com/paper/2102.00290/) (2021)
- [Domain Adaptation for Sparse-Data Settings: What Do We Gain by Not Using Bert?](https://mlsanity.com/paper/2203.16926/) (2022)

## Abstract (arXiv)

In this paper, we focus on the problem of adapting word vector-based models to new textual data. Given a model pre-trained on large reference data, how can we adapt it to a smaller piece of data with a slightly different language distribution? We frame the adaptation problem as a monolingual word vector alignment problem, and simply average models after alignment. We align vectors using the RCSLS criterion. Our formulation results in a simple and efficient algorithm that allows adapting general-purpose models to changing word distributions. In our evaluation, we consider applications to word embedding and text classification models. We show that the proposed approach yields good performance in all setups and outperforms a baseline consisting in fine-tuning the model on new data.