# Evaluation of Morphological Embeddings for the Russian Language

arXiv: https://arxiv.org/abs/2103.06628 · HTML: https://mlsanity.com/paper/2103.06628/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 11 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [A Survey of Word Embeddings Evaluation Methods](https://mlsanity.com/paper/1801.09536/) (2018)
- [Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations](https://mlsanity.com/paper/1808.09500/) (2018)
- [A Joint Model for Word Embedding and Word Morphology](https://mlsanity.com/paper/1606.02601/) (2016)
- [Compositional Morphology for Word Representations and Language Modelling](https://mlsanity.com/paper/1405.4273/) (2014)
- [Morphological Word Embeddings](https://mlsanity.com/paper/1907.02423/) (2019)
- [Morphological Priors for Probabilistic Neural Word Embeddings](https://mlsanity.com/paper/1608.01056/) (2016)
- [The Interplay of Semantics and Morphology in Word Embeddings](https://mlsanity.com/paper/1704.01938/) (2017)

## Abstract (arXiv)

A number of morphology-based word embedding models were introduced in recent years. However, their evaluation was mostly limited to English, which is known to be a morphologically simple language. In this paper, we explore whether and to what extent incorporating morphology into word embeddings improves performance on downstream NLP tasks, in the case of morphologically rich Russian language. NLP tasks of our choice are POS tagging, Chunking, and NER -- for Russian language, all can be mostly solved using only morphology without understanding the semantics of words. Our experiments show that morphology-based embeddings trained with Skipgram objective do not outperform existing embedding model -- FastText. Moreover, a more complex, but morphology unaware model, BERT, allows to achieve significantly greater performance on the tasks that presumably require understanding of a word's morphology.