# Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter

arXiv: https://arxiv.org/abs/1811.02906 · HTML: https://mlsanity.com/paper/1811.02906/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2018.

## Built on by (most cited first, 6 of 6)

- [Ruddit: Norms of Offensiveness for English Reddit Comments](https://mlsanity.com/paper/2106.05664/) (2021)
- [Multi-Task Learning with Sentiment, Emotion, and Target Detection to Recognize Hate Speech and Offensive Language](https://mlsanity.com/paper/2109.10255/) (2021)
- [Detecting Text Formality: A Study of Text Classification Approaches](https://mlsanity.com/paper/2204.08975/) (2022)
- [Transferability analysis of data-driven additive manufacturing knowledge: a case study between powder bed fusion and directed energy deposition](https://mlsanity.com/paper/2309.06286/) (2023)
- [Addressing the Challenges of Cross-Lingual Hate Speech Detection](https://mlsanity.com/paper/2201.05922/) (2022)
- [Detection of Homophobia & Transphobia in Dravidian Languages: Exploring Deep Learning Methods](https://mlsanity.com/paper/2304.01241/) (2023)

## Abstract (arXiv)

We investigate different strategies for automatic offensive language classification on German Twitter data. For this, we employ a sequentially combined BiLSTM-CNN neural network. Based on this model, three transfer learning tasks to improve the classification performance with background knowledge are tested. We compare 1. Supervised category transfer: social media data annotated with near-offensive language categories, 2. Weakly-supervised category transfer: tweets annotated with emojis they contain, 3. Unsupervised category transfer: tweets annotated with topic clusters obtained by Latent Dirichlet Allocation (LDA). Further, we investigate the effect of three different strategies to mitigate negative effects of 'catastrophic forgetting' during transfer learning. Our results indicate that transfer learning in general improves offensive language detection. Best results are achieved from pre-training our model on the unsupervised topic clustering of tweets in combination with thematic user cluster information.