# Lifelong Learning of Hate Speech Classification on Social Media

arXiv: https://arxiv.org/abs/2106.02821 · HTML: https://mlsanity.com/paper/2106.02821/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1805.07810/) (2018)
- [Episodic Memory in Lifelong Language Learning](https://mlsanity.com/paper/1906.01076/) (2019)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [LAMOL: LAnguage MOdeling for Lifelong Language Learning](https://mlsanity.com/paper/1909.03329/) (2019)
- [Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting](https://mlsanity.com/paper/1802.02950/) (2018)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection](https://mlsanity.com/paper/1804.03124/) (2018)
- [Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention](https://mlsanity.com/paper/1605.09090/) (2016)
- [Recognizing Explicit and Implicit Hate Speech Using a Weakly Supervised Two-path Bootstrapping Approach](https://mlsanity.com/paper/1710.07394/) (2017)
- [Hierarchical CVAE for Fine-Grained Hate Speech Classification](https://mlsanity.com/paper/1809.00088/) (2018)

## Built on by (most cited first, 14 of 14)

- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [Towards Lifelong Learning of Large Language Models: A Survey](https://mlsanity.com/paper/2406.06391/) (2024)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models](https://mlsanity.com/paper/2310.07589/) (2023)
- [KOLD: Korean Offensive Language Dataset](https://mlsanity.com/paper/2205.11315/) (2022)
- [Revisiting Hate Speech Benchmarks: From Data Curation to System Deployment](https://mlsanity.com/paper/2306.01105/) (2023)
- [The State of Profanity Obfuscation in Natural Language Processing](https://mlsanity.com/paper/2210.07595/) (2022)
- [Hatemongers ride on echo chambers to escalate hate speech diffusion](https://mlsanity.com/paper/2302.02479/) (2023)
- [Anti-Asian Hate Speech Detection via Data Augmented Semantic Relation Inference](https://mlsanity.com/paper/2204.07010/) (2022)
- [Towards a Unified Framework for Adaptable Problematic Content Detection via Continual Learning](https://mlsanity.com/paper/2309.16905/) (2023)
- [Going Extreme: Comparative Analysis of Hate Speech in Parler and Gab](https://mlsanity.com/paper/2201.11770/) (2022)
- [DoDo Learning: DOmain-DemOgraphic Transfer in Language Models for Detecting Abuse Targeted at Public Figures](https://mlsanity.com/paper/2307.16811/) (2023)
- [Hatevolution: What Static Benchmarks Don't Tell Us](https://mlsanity.com/paper/2506.12148/) (2025)
- [Knowledge-guided Continual Learning for Behavioral Analytics Systems](https://mlsanity.com/paper/2510.22405/) (2025)

## Abstract (arXiv)

Existing work on automated hate speech classification assumes that the dataset is fixed and the classes are pre-defined. However, the amount of data in social media increases every day, and the hot topics changes rapidly, requiring the classifiers to be able to continuously adapt to new data without forgetting the previously learned knowledge. This ability, referred to as lifelong learning, is crucial for the real-word application of hate speech classifiers in social media. In this work, we propose lifelong learning of hate speech classification on social media. To alleviate catastrophic forgetting, we propose to use Variational Representation Learning (VRL) along with a memory module based on LB-SOINN (Load-Balancing Self-Organizing Incremental Neural Network). Experimentally, we show that combining variational representation learning and the LB-SOINN memory module achieves better performance than the commonly-used lifelong learning techniques.