# Addressing machine learning concept drift reveals declining vaccine sentiment during the COVID-19 pandemic

arXiv: https://arxiv.org/abs/2012.02197 · HTML: https://mlsanity.com/paper/2012.02197/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 5 of 5)

- [Tackling Concept Shift in Text Classification using Entailment-style Modeling](https://mlsanity.com/paper/2311.03320/) (2023)
- [Interpretable Model Drift Detection](https://mlsanity.com/paper/2503.06606/) (2025)
- [The Silent Problem -- Machine Learning Model Failure -- How to Diagnose and Fix Ailing Machine Learning Models](https://mlsanity.com/paper/2204.10227/) (2022)
- [No Time Like the Present: Effects of Language Change on Automated Comment Moderation](https://mlsanity.com/paper/2207.04003/) (2022)
- [A Holistic Indicator of Polarization to Measure Online Sexism](https://mlsanity.com/paper/2404.02205/) (2024)

## Abstract (arXiv)

Social media analysis has become a common approach to assess public opinion on various topics, including those about health, in near real-time. The growing volume of social media posts has led to an increased usage of modern machine learning methods in natural language processing. While the rapid dynamics of social media can capture underlying trends quickly, it also poses a technical problem: algorithms trained on annotated data in the past may underperform when applied to contemporary data. This phenomenon, known as concept drift, can be particularly problematic when rapid shifts occur either in the topic of interest itself, or in the way the topic is discussed. Here, we explore the effect of machine learning concept drift by focussing on vaccine sentiments expressed on Twitter, a topic of central importance especially during the COVID-19 pandemic. We show that while vaccine sentiment has declined considerably during the COVID-19 pandemic in 2020, algorithms trained on pre-pandemic data would have largely missed this decline due to concept drift. Our results suggest that social media analysis systems must address concept drift in a continuous fashion in order to avoid the risk of systematic misclassification of data, which is particularly likely during a crisis when the underlying data can change suddenly and rapidly.