# TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Media

arXiv: https://arxiv.org/abs/2209.07216 · HTML: https://mlsanity.com/paper/2209.07216/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [TimeLMs: Diachronic Language Models from Twitter](https://mlsanity.com/paper/2202.03829/) (2022)
- [Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora](https://mlsanity.com/paper/2110.08534/) (2021)
- [Time Waits for No One! Analysis and Challenges of Temporal Misalignment](https://mlsanity.com/paper/2111.07408/) (2021)
- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Temporal Effects on Pre-trained Models for Language Processing Tasks](https://mlsanity.com/paper/2111.12790/) (2021)
- [XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization](https://mlsanity.com/paper/2010.06478/) (2020)
- [Short-Term Meaning Shift: A Distributional Exploration](https://mlsanity.com/paper/1809.03169/) (2018)
- [Dynamic Contextualized Word Embeddings](https://mlsanity.com/paper/2010.12684/) (2020)
- [LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond](https://mlsanity.com/paper/2105.12449/) (2021)
- [AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples](https://mlsanity.com/paper/2104.08639/) (2021)
- [WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context](https://mlsanity.com/paper/2004.15016/) (2020)
- [Mitigating Temporal-Drift: A Simple Approach to Keep NER Models Crisp](https://mlsanity.com/paper/2104.09742/) (2021)
- [Exploring Emoji Usage and Prediction Through a Temporal Variation Lens](https://mlsanity.com/paper/1805.00731/) (2018)

## Built on by (most cited first, 19 of 19)

- [CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation](https://mlsanity.com/paper/2310.15638/) (2023)
- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](https://mlsanity.com/paper/2402.12011/) (2024)
- [Evaluating Decision Models for Text Annotation in Computational Social Science](https://mlsanity.com/paper/2609.24574/) (2026)
- [SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research](https://mlsanity.com/paper/2310.14757/) (2023)
- [(Chat)GPT v BERT: Dawn of Justice for Semantic Change Detection](https://mlsanity.com/paper/2401.14040/) (2024)
- [CUB: Benchmarking Context Utilisation Techniques for Language Models](https://mlsanity.com/paper/2505.16518/) (2025)
- [Tweet Insights: A Visualization Platform to Extract Temporal Insights from Twitter](https://mlsanity.com/paper/2308.02142/) (2023)
- [The LSCD Benchmark: a Testbed for Diachronic Word Meaning Tasks](https://mlsanity.com/paper/2404.00176/) (2024)
- [Meaning at the Planck scale? Contextualized word embeddings for doing history, philosophy, and sociology of science](https://mlsanity.com/paper/2411.14073/) (2024)
- [Is It Worth the (Environmental) Cost? Limited Evidence for Temporal Adaptation via Continuous Training](https://mlsanity.com/paper/2210.07365/) (2022)
- [Using Deep Mixture-of-Experts to Detect Word Meaning Shift for TempoWiC](https://mlsanity.com/paper/2211.03466/) (2022)
- [Large Language Models on Lexical Semantic Change Detection: An Evaluation](https://mlsanity.com/paper/2312.06002/) (2023)
- [LSC-Eval: A General Framework to Evaluate Methods for Assessing Dimensions of Lexical Semantic Change Using LLM-Generated Synthetic Data](https://mlsanity.com/paper/2503.08042/) (2025)
- [Temporal Word Meaning Disambiguation using TimeLMs](https://mlsanity.com/paper/2210.08207/) (2022)
- [CHEW: A Dataset of CHanging Events in Wikipedia](https://mlsanity.com/paper/2406.19116/) (2024)
- [Evaluating the Evaluator: Problems with SemEval-2020 Task 1 for Lexical Semantic Change Detection](https://mlsanity.com/paper/2604.13232/) (2026)
- [Using Two Losses and Two Datasets Simultaneously to Improve TempoWiC Accuracy](https://mlsanity.com/paper/2212.07669/) (2022)
- [A Systematic Analysis on the Temporal Generalization of Language Models in Social Media](https://mlsanity.com/paper/2405.13017/) (2024)

## Abstract (arXiv)

Language evolves over time, and word meaning changes accordingly. This is especially true in social media, since its dynamic nature leads to faster semantic shifts, making it challenging for NLP models to deal with new content and trends. However, the number of datasets and models that specifically address the dynamic nature of these social platforms is scarce. To bridge this gap, we present TempoWiC, a new benchmark especially aimed at accelerating research in social media-based meaning shift. Our results show that TempoWiC is a challenging benchmark, even for recently-released language models specialized in social media.