# A Joint Model for Word Embedding and Word Morphology

arXiv: https://arxiv.org/abs/1606.02601 · HTML: https://mlsanity.com/paper/1606.02601/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2016.

## Built on by (most cited first, 14 of 14)

- [You should evaluate your language model on marginal likelihood over tokenisations](https://mlsanity.com/paper/2109.02550/) (2021)
- [Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost](https://mlsanity.com/paper/2203.07860/) (2022)
- [What is the best recipe for character-level encoder-only modelling?](https://mlsanity.com/paper/2305.05461/) (2023)
- [A Survey of Text Representation Methods and Their Genealogy](https://mlsanity.com/paper/2211.14591/) (2022)
- [Automated Query Reformulation for Efficient Search based on Query Logs From Stack Overflow](https://mlsanity.com/paper/2102.00826/) (2021)
- [Whois? Deep Author Name Disambiguation using Bibliographic Data](https://mlsanity.com/paper/2207.04772/) (2022)
- [Inducing Meaningful Units from Character Sequences with Dynamic Capacity Slot Attention](https://mlsanity.com/paper/2102.01223/) (2021)
- [A Survey On Neural Word Embeddings](https://mlsanity.com/paper/2110.01804/) (2021)
- [A Quantitative and Qualitative Analysis of Suicide Ideation Detection using Deep Learning](https://mlsanity.com/paper/2206.08673/) (2022)
- [Bib2Auth: Deep Learning Approach for Author Disambiguation using Bibliographic Data](https://mlsanity.com/paper/2107.04382/) (2021)
- [Evaluation of Morphological Embeddings for the Russian Language](https://mlsanity.com/paper/2103.06628/) (2021)
- [Evaluation of Morphological Embeddings for English and Russian Languages](https://mlsanity.com/paper/2103.06884/) (2021)
- [Explainable Sentence-Level Sentiment Analysis for Amazon Product Reviews](https://mlsanity.com/paper/2111.06070/) (2021)
- [Deep Author Name Disambiguation using DBLP Data](https://mlsanity.com/paper/2303.10067/) (2023)

## Abstract (arXiv)

This paper presents a joint model for performing unsupervised morphological analysis on words, and learning a character-level composition function from morphemes to word embeddings. Our model splits individual words into segments, and weights each segment according to its ability to predict context words. Our morphological analysis is comparable to dedicated morphological analyzers at the task of morpheme boundary recovery, and also performs better than word-based embedding models at the task of syntactic analogy answering. Finally, we show that incorporating morphology explicitly into character-level models help them produce embeddings for unseen words which correlate better with human judgments.