# No Time Like the Present: Effects of Language Change on Automated Comment Moderation

arXiv: https://arxiv.org/abs/2207.04003 · HTML: https://mlsanity.com/paper/2207.04003/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 59 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Directions in Abusive Language Training Data: Garbage In, Garbage Out](https://mlsanity.com/paper/2004.01670/) (2020)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [The Social Dynamics of Language Change in Online Networks](https://mlsanity.com/paper/1609.02075/) (2016)
- [Addressing machine learning concept drift reveals declining vaccine sentiment during the COVID-19 pandemic](https://mlsanity.com/paper/2012.02197/) (2020)

## Abstract (arXiv)

The spread of online hate has become a significant problem for newspapers that host comment sections. As a result, there is growing interest in using machine learning and natural language processing for (semi-) automated abusive language detection to avoid manual comment moderation costs or having to shut down comment sections altogether. However, much of the past work on abusive language detection assumes that classifiers operate in a static language environment, despite language and news being in a state of constant flux. In this paper, we show using a new German newspaper comments dataset that the classifiers trained with naive ML techniques like a random-test train split will underperform on future data, and that a time stratified evaluation split is more appropriate. We also show that classifier performance rapidly degrades when evaluated on data from a different period than the training data. Our findings suggest that it is necessary to consider the temporal dynamics of language when developing an abusive language detection system or risk deploying a model that will quickly become defunct.