# Training Temporal Word Embeddings with a Compass

arXiv: https://arxiv.org/abs/1906.02376 · HTML: https://mlsanity.com/paper/1906.02376/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 23 of 23)

- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research](https://mlsanity.com/paper/2412.04497/) (2024)
- [Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases](https://mlsanity.com/paper/2108.06295/) (2021)
- [Opinions are Made to be Changed: Temporally Adaptive Stance Classification](https://mlsanity.com/paper/2108.12476/) (2021)
- [SWEAT: Scoring Polarization of Topics across Different Corpora](https://mlsanity.com/paper/2109.07231/) (2021)
- [DRIFT: A Toolkit for Diachronic Analysis of Scientific Literature](https://mlsanity.com/paper/2107.01198/) (2021)
- [A Greek Parliament Proceedings Dataset for Computational Linguistics and Political Analysis](https://mlsanity.com/paper/2210.12883/) (2022)
- [Turkronicles: Diachronic Resources for the Fast Evolving Turkish Language](https://mlsanity.com/paper/2405.10133/) (2024)
- [Tweet Insights: A Visualization Platform to Extract Temporal Insights from Twitter](https://mlsanity.com/paper/2308.02142/) (2023)
- [Syntactic Language Change in English and German: Metrics, Parsers, and Convergences](https://mlsanity.com/paper/2402.11549/) (2024)
- [Computational modeling of semantic change](https://mlsanity.com/paper/2304.06337/) (2023)
- [A Systematic Survey of Natural Language Processing for the Greek Language](https://mlsanity.com/paper/2407.09861/) (2024)
- [Visualizing Temporal Topic Embeddings with a Compass](https://mlsanity.com/paper/2409.10649/) (2024)
- [Metaphors' journeys across time and genre: tracking the evolution of literary metaphors with temporal embeddings](https://mlsanity.com/paper/2602.13701/) (2026)
- [Towards Using Diachronic Distributed Word Representations as Models of Lexical Development](https://mlsanity.com/paper/2105.05091/) (2021)
- [Diachronic Text Mining Investigation of Therapeutic Candidates for COVID-19](https://mlsanity.com/paper/2110.13971/) (2021)
- [On the Impact of Temporal Representations on Metaphor Detection](https://mlsanity.com/paper/2111.03320/) (2021)
- [Characterizing Human Actions in the Digital Platform by Temporal Context](https://mlsanity.com/paper/2206.09535/) (2022)
- [Trust in Motion: Capturing Trust Ascendancy in Open-Source Projects using Hybrid AI](https://mlsanity.com/paper/2210.02656/) (2022)
- [Values That Are Explicitly Present in Fairy Tales: Comparing Samples from German, Italian and Portuguese Traditions](https://mlsanity.com/paper/2402.08318/) (2024)
- [Detecting Turkish Synonyms Used in Different Time Periods](https://mlsanity.com/paper/2411.15768/) (2024)
- [Beyond frequency measures: Can contextual embeddings capture meaning change in scientific texts?](https://mlsanity.com/paper/2609.18804/) (2026)
- [Lexical Persistence Without Semantic Entrenchment in AI Agent Communities](https://mlsanity.com/paper/2610.20365/) (2026)

## Abstract (arXiv)

Temporal word embeddings have been proposed to support the analysis of word meaning shifts during time and to study the evolution of languages. Different approaches have been proposed to generate vector representations of words that embed their meaning during a specific time interval. However, the training process used in these approaches is complex, may be inefficient or it may require large text corpora. As a consequence, these approaches may be difficult to apply in resource-scarce domains or by scientists with limited in-depth knowledge of embedding models. In this paper, we propose a new heuristic to train temporal word embeddings based on the Word2vec model. The heuristic consists in using atemporal vectors as a reference, i.e., as a compass, when training the representations specific to a given time interval. The use of the compass simplifies the training process and makes it more efficient. Experiments conducted using state-of-the-art datasets and methodologies suggest that our approach outperforms or equals comparable approaches while being more robust in terms of the required corpus size.