# Morphological Priors for Probabilistic Neural Word Embeddings

arXiv: https://arxiv.org/abs/1608.01056 · HTML: https://mlsanity.com/paper/1608.01056/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2016.

## Built on by (most cited first, 6 of 6)

- [Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words](https://mlsanity.com/paper/2101.00403/) (2021)
- [MorphTE: Injecting Morphology in Tensorized Embeddings](https://mlsanity.com/paper/2210.15379/) (2022)
- [Integrating Approaches to Word Representation](https://mlsanity.com/paper/2109.04876/) (2021)
- [From Characters to Words: Hierarchical Pre-trained Language Model for Open-vocabulary Language Understanding](https://mlsanity.com/paper/2305.14571/) (2023)
- [Evaluation of Morphological Embeddings for the Russian Language](https://mlsanity.com/paper/2103.06628/) (2021)
- [Evaluation of Morphological Embeddings for English and Russian Languages](https://mlsanity.com/paper/2103.06884/) (2021)

## Abstract (arXiv)

Word embeddings allow natural language processing systems to share statistical information across related words. These embeddings are typically based on distributional statistics, making it difficult for them to generalize to rare or unseen words. We propose to improve word embeddings by incorporating morphological information, capturing shared sub-word features. Unlike previous work that constructs word embeddings directly from morphemes, we combine morphological and distributional information in a unified probabilistic framework, in which the word embedding is a latent variable. The morphological information provides a prior distribution on the latent word embeddings, which in turn condition a likelihood function over an observed corpus. This approach yields improvements on intrinsic word similarity evaluations, and also in the downstream task of part-of-speech tagging.