# Lifelong Domain Word Embedding via Meta-Learning

arXiv: https://arxiv.org/abs/1805.09991 · HTML: https://mlsanity.com/paper/1805.09991/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2018.

## Built on by (most cited first, 8 of 8)

- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [Deep Bayesian Unsupervised Lifelong Learning](https://mlsanity.com/paper/2106.07035/) (2021)
- [Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through Rate Prediction](https://mlsanity.com/paper/2105.08909/) (2021)
- [Studying Catastrophic Forgetting in Neural Ranking Models](https://mlsanity.com/paper/2101.06984/) (2021)
- [Life-long Learning for Multilingual Neural Machine Translation with Knowledge Distillation](https://mlsanity.com/paper/2212.02800/) (2022)
- [TransDrift: Modeling Word-Embedding Drift using Transformer](https://mlsanity.com/paper/2206.08081/) (2022)
- [Coherence-Based Distributed Document Representation Learning for Scientific Documents](https://mlsanity.com/paper/2201.02846/) (2022)
- [CONE: Embeddings for Complex Numerical Data Preserving Unit and Variable Semantics](https://mlsanity.com/paper/2603.04741/) (2026)

## Abstract (arXiv)

Learning high-quality domain word embeddings is important for achieving good performance in many NLP tasks. General-purpose embeddings trained on large-scale corpora are often sub-optimal for domain-specific applications. However, domain-specific tasks often do not have large in-domain corpora for training high-quality domain embeddings. In this paper, we propose a novel lifelong learning setting for domain embedding. That is, when performing the new domain embedding, the system has seen many past domains, and it tries to expand the new in-domain corpus by exploiting the corpora from the past domains via meta-learning. The proposed meta-learner characterizes the similarities of the contexts of the same word in many domain corpora, which helps retrieve relevant data from the past domains to expand the new domain corpus. Experimental results show that domain embeddings produced from such a process improve the performance of the downstream tasks.