# Deep-change at AXOLOTL-24: Orchestrating WSD and WSI Models for Semantic Change Modeling

arXiv: https://arxiv.org/abs/2408.05184 · HTML: https://mlsanity.com/paper/2408.05184/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 9 Aug 2024.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [Explaining and Improving BERT Performance on Lexical Semantic Change Detection](https://mlsanity.com/paper/2103.07259/) (2021)
- [Morphological Analyzer and Generator for Russian and Ukrainian Languages](https://mlsanity.com/paper/1503.07283/) (2015)
- [Word Sense Induction with Neural biLM and Symmetric Patterns](https://mlsanity.com/paper/1808.08518/) (2018)
- [Towards better substitution-based word sense induction](https://mlsanity.com/paper/1905.12598/) (2019)
- [Semeval-2022 Task 1: CODWOE -- Comparing Dictionaries and Word Embeddings](https://mlsanity.com/paper/2205.13858/) (2022)
- [Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution](https://mlsanity.com/paper/2206.11815/) (2022)
- [Graph-based Clustering for Detecting Semantic Change Across Time and Languages](https://mlsanity.com/paper/2402.01025/) (2024)
- [CIRCE at SemEval-2020 Task 1: Ensembling Context-Free and Context-Dependent Word Representations](https://mlsanity.com/paper/2005.06602/) (2020)
- [UWB at SemEval-2020 Task 1: Lexical Semantic Change Detection](https://mlsanity.com/paper/2012.00004/) (2020)
- [Detection of Non-recorded Word Senses in English and Swedish](https://mlsanity.com/paper/2403.02285/) (2024)
- [AXOLOTL'24 Shared Task on Multilingual Explainable Semantic Change Modeling](https://mlsanity.com/paper/2407.04079/) (2024)
- [Multilingual Substitution-based Word Sense Induction](https://mlsanity.com/paper/2405.11086/) (2024)

## Abstract (arXiv)

This paper describes our solution of the first subtask from the AXOLOTL-24 shared task on Semantic Change Modeling. The goal of this subtask is to distribute a given set of usages of a polysemous word from a newer time period between senses of this word from an older time period and clusters representing gained senses of this word. We propose and experiment with three new methods solving this task. Our methods achieve SOTA results according to both official metrics of the first substask. Additionally, we develop a model that can tell if a given word usage is not described by any of the provided sense definitions. This model serves as a component in one of our methods, but can potentially be useful on its own.