# Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection

arXiv: https://arxiv.org/abs/2004.13432 · HTML: https://mlsanity.com/paper/2004.13432/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2020.

## Built on by (most cited first, 12 of 12)

- [Multimodal End-to-End Sparse Model for Emotion Recognition](https://mlsanity.com/paper/2103.09666/) (2021)
- [Weakly-supervised Multi-task Learning for Multimodal Affect Recognition](https://mlsanity.com/paper/2104.11560/) (2021)
- [WLV-RIT at GermEval 2021: Multitask Learning with Transformers to Detect Toxic, Engaging, and Fact-Claiming Comments](https://mlsanity.com/paper/2108.00057/) (2021)
- [Cisco at SemEval-2021 Task 5: What's Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Comments](https://mlsanity.com/paper/2105.13959/) (2021)
- [Offensive Language Detection with BERT-based models, By Customizing Attention Probabilities](https://mlsanity.com/paper/2110.05133/) (2021)
- [Text generation for dataset augmentation in security classification tasks](https://mlsanity.com/paper/2310.14429/) (2023)
- [Deep Prompt Multi-task Network for Abuse Language Detection](https://mlsanity.com/paper/2403.05268/) (2024)
- [A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings](https://mlsanity.com/paper/2505.12116/) (2025)
- [Detecting Offensive Language on Social Networks: An End-to-end Detection Method based on Graph Attention Networks](https://mlsanity.com/paper/2203.02123/) (2022)
- [Kaggle Competition: Cantonese Audio-Visual Speech Recognition for In-car Commands](https://mlsanity.com/paper/2207.02663/) (2022)
- [U-GIFT: Uncertainty-Guided Firewall for Toxic Speech in Few-Shot Scenario](https://mlsanity.com/paper/2501.00907/) (2025)
- [Cascading versus Joint Modeling for Hierarchical Offensive Language Detection](https://mlsanity.com/paper/2607.16790/) (2026)

## Abstract (arXiv)

Nowadays, offensive content in social media has become a serious problem, and automatically detecting offensive language is an essential task. In this paper, we build an offensive language detection system, which combines multi-task learning with BERT-based models. Using a pre-trained language model such as BERT, we can effectively learn the representations for noisy text in social media. Besides, to boost the performance of offensive language detection, we leverage the supervision signals from other related tasks. In the OffensEval-2020 competition, our model achieves 91.51% F1 score in English Sub-task A, which is comparable to the first place (92.23%F1). An empirical analysis is provided to explain the effectiveness of our approaches.