# Compositional Morphology for Word Representations and Language Modelling

arXiv: https://arxiv.org/abs/1405.4273 · HTML: https://mlsanity.com/paper/1405.4273/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2014.

## Built on by (most cited first, 11 of 11)

- [Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words](https://mlsanity.com/paper/2101.00403/) (2021)
- [The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?](https://mlsanity.com/paper/2601.07220/) (2026)
- [A Natural Bias for Language Generation Models](https://mlsanity.com/paper/2212.09686/) (2022)
- [MorphTE: Injecting Morphology in Tensorized Embeddings](https://mlsanity.com/paper/2210.15379/) (2022)
- [Integrating Approaches to Word Representation](https://mlsanity.com/paper/2109.04876/) (2021)
- [A Survey On Neural Word Embeddings](https://mlsanity.com/paper/2110.01804/) (2021)
- [When FastText Pays Attention: Efficient Estimation of Word Representations using Constrained Positional Weighting](https://mlsanity.com/paper/2104.09691/) (2021)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation](https://mlsanity.com/paper/2210.15358/) (2022)
- [Evaluation of Morphological Embeddings for the Russian Language](https://mlsanity.com/paper/2103.06628/) (2021)
- [Evaluation of Morphological Embeddings for English and Russian Languages](https://mlsanity.com/paper/2103.06884/) (2021)

## Abstract (arXiv)

This paper presents a scalable method for integrating compositional morphological representations into a vector-based probabilistic language model. Our approach is evaluated in the context of log-bilinear language models, rendered suitably efficient for implementation inside a machine translation decoder by factoring the vocabulary. We perform both intrinsic and extrinsic evaluations, presenting results on a range of languages which demonstrate that our model learns morphological representations that both perform well on word similarity tasks and lead to substantial reductions in perplexity. When used for translation into morphologically rich languages with large vocabularies, our models obtain improvements of up to 1.2 BLEU points relative to a baseline system using back-off n-gram models.