# WLV-RIT at GermEval 2021: Multitask Learning with Transformers to Detect Toxic, Engaging, and Fact-Claiming Comments

arXiv: https://arxiv.org/abs/2108.00057 · HTML: https://mlsanity.com/paper/2108.00057/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 30 Jul 2021.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [German's Next Language Model](https://mlsanity.com/paper/2010.10906/) (2020)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Multilingual Offensive Language Identification with Cross-lingual Embeddings](https://mlsanity.com/paper/2010.05324/) (2020)
- [Offensive Language Identification in Greek](https://mlsanity.com/paper/2003.07459/) (2020)
- [Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech](https://mlsanity.com/paper/2106.01625/) (2021)
- [MUDES: Multilingual Detection of Offensive Spans](https://mlsanity.com/paper/2102.09665/) (2021)
- [Arabic Offensive Language on Twitter: Analysis and Experiments](https://mlsanity.com/paper/2004.02192/) (2020)
- [Ruddit: Norms of Offensiveness for English Reddit Comments](https://mlsanity.com/paper/2106.05664/) (2021)
- [Challenges in Discriminating Profanity from Hate Speech](https://mlsanity.com/paper/1803.05495/) (2018)
- [Multilingual Offensive Language Identification for Low-resource Languages](https://mlsanity.com/paper/2105.05996/) (2021)
- [An Exploratory Analysis of the Relation Between Offensive Language and Mental Health](https://mlsanity.com/paper/2105.14888/) (2021)
- [Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection](https://mlsanity.com/paper/2004.13432/) (2020)
- [Comparing Approaches to Dravidian Language Identification](https://mlsanity.com/paper/2103.05552/) (2021)
- [Fact Checking in Community Forums](https://mlsanity.com/paper/1803.03178/) (2018)
- [InfoMiner at WNUT-2020 Task 2: Transformer-based Covid-19 Informative Tweet Extraction](https://mlsanity.com/paper/2010.05327/) (2020)
- [BRUMS at SemEval-2020 Task 3: Contextualised Embeddings for Predicting the (Graded) Effect of Context in Word Similarity](https://mlsanity.com/paper/2010.06269/) (2020)

## Built on by (most cited first, 3 of 3)

- [Same Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection](https://mlsanity.com/paper/2302.07372/) (2023)
- [BERT(s) to Detect Multiword Expressions](https://mlsanity.com/paper/2208.07832/) (2022)
- [Can Model Fusing Help Transformers in Long Document Classification? An Empirical Study](https://mlsanity.com/paper/2307.09532/) (2023)

## Abstract (arXiv)

This paper addresses the identification of toxic, engaging, and fact-claiming comments on social media. We used the dataset made available by the organizers of the GermEval-2021 shared task containing over 3,000 manually annotated Facebook comments in German. Considering the relatedness of the three tasks, we approached the problem using large pre-trained transformer models and multitask learning. Our results indicate that multitask learning achieves performance superior to the more common single task learning approach in all three tasks. We submit our best systems to GermEval-2021 under the team name WLV-RIT.