# The Problem of Semantic Shift in Longitudinal Monitoring of Social Media: A Case Study on Mental Health During the COVID-19 Pandemic

arXiv: https://arxiv.org/abs/2206.11160 · HTML: https://mlsanity.com/paper/2206.11160/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 103 references that are in the map.
- First posted on arXiv 22 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [TimeLMs: Diachronic Language Models from Twitter](https://mlsanity.com/paper/2202.03829/) (2022)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Understanding the Origins of Bias in Word Embeddings](https://mlsanity.com/paper/1810.03611/) (2018)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [Multi-Task Learning for Mental Health using Social Media Text](https://mlsanity.com/paper/1712.03538/) (2017)
- [Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora](https://mlsanity.com/paper/2112.14330/) (2021)
- [Gender and Racial Fairness in Depression Research using Social Media](https://mlsanity.com/paper/2103.10550/) (2021)
- [Correcting Sociodemographic Selection Biases for Population Prediction from Social Media](https://mlsanity.com/paper/1911.03855/) (2019)
- [Quantifying the Effects of COVID-19 on Mental Health Support Forums](https://mlsanity.com/paper/2009.04008/) (2020)
- [Can We Assess Mental Health through Social Media and Smart Devices? Addressing Bias in Methodology and Evaluation](https://mlsanity.com/paper/1807.07351/) (2018)
- [Temporal Mental Health Dynamics on Social Media](https://mlsanity.com/paper/2008.13121/) (2020)
- [SCoT: Sense Clustering over Time: a tool for the analysis of lexical change](https://mlsanity.com/paper/2203.09892/) (2022)

## Built on by (most cited first, 3 of 3)

- [Exploring Social Media for Early Detection of Depression in COVID-19 Patients](https://mlsanity.com/paper/2302.12044/) (2023)
- [Leveraging Prompt-Based Large Language Models: Predicting Pandemic Health Decisions and Outcomes Through Social Media Language](https://mlsanity.com/paper/2403.00994/) (2024)
- [On the State of NLP Approaches to Modeling Depression in Social Media: A Post-COVID-19 Outlook](https://mlsanity.com/paper/2410.08793/) (2024)

## Abstract (arXiv)

Social media allows researchers to track societal and cultural changes over time based on language analysis tools. Many of these tools rely on statistical algorithms which need to be tuned to specific types of language. Recent studies have shown the absence of appropriate tuning, specifically in the presence of semantic shift, can hinder robustness of the underlying methods. However, little is known about the practical effect this sensitivity may have on downstream longitudinal analyses. We explore this gap in the literature through a timely case study: understanding shifts in depression during the course of the COVID-19 pandemic. We find that inclusion of only a small number of semantically-unstable features can promote significant changes in longitudinal estimates of our target outcome. At the same time, we demonstrate that a recently-introduced method for measuring semantic shift may be used to proactively identify failure points of language-based models and, in turn, improve predictive generalization.