# ELMo and BERT in semantic change detection for Russian

arXiv: https://arxiv.org/abs/2010.03481 · HTML: https://mlsanity.com/paper/2010.03481/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](https://mlsanity.com/paper/2402.12011/) (2024)
- [Evaluation of contextual embeddings on less-resourced languages](https://mlsanity.com/paper/2107.10614/) (2021)
- [On the Universality of Deep Contextual Language Models](https://mlsanity.com/paper/2109.07140/) (2021)
- [Semantic Change Detection for the Romanian Language](https://mlsanity.com/paper/2308.12131/) (2023)
- [Tracking Semantic Change in Slovene: A Novel Dataset and Optimal Transport-Based Distance](https://mlsanity.com/paper/2402.16596/) (2024)
- [Comparative Analysis of Static and Contextual Embeddings for Analyzing Semantic Changes in Medieval Latin Charters](https://mlsanity.com/paper/2410.09283/) (2024)
- [Dynamics of Meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhala](https://mlsanity.com/paper/2609.08609/) (2026)

## Abstract (arXiv)

We study the effectiveness of contextualized embeddings for the task of diachronic semantic change detection for Russian language data. Evaluation test sets consist of Russian nouns and adjectives annotated based on their occurrences in texts created in pre-Soviet, Soviet and post-Soviet time periods. ELMo and BERT architectures are compared on the task of ranking Russian words according to the degree of their semantic change over time. We use several methods for aggregation of contextualized embeddings from these architectures and evaluate their performance. Finally, we compare unsupervised and supervised techniques in this task.