# Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification using Pre-trained Language Models

arXiv: https://arxiv.org/abs/2010.03542 · HTML: https://mlsanity.com/paper/2010.03542/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [A Primer on Pretrained Multilingual Language Models](https://mlsanity.com/paper/2107.00676/) (2021)
- [MUDES: Multilingual Detection of Offensive Spans](https://mlsanity.com/paper/2102.09665/) (2021)
- [A systematic review of Hate Speech automatic detection using Natural Language Processing](https://mlsanity.com/paper/2106.00742/) (2021)
- [Multilingual Offensive Language Identification for Low-resource Languages](https://mlsanity.com/paper/2105.05996/) (2021)
- [Distilling Knowledge from Large Language Models: A Concept Bottleneck Model for Hate and Counter Speech Recognition](https://mlsanity.com/paper/2508.08274/) (2025)
- [Offensive Language Detection with BERT-based models, By Customizing Attention Probabilities](https://mlsanity.com/paper/2110.05133/) (2021)
- [Language Model Detoxification in Dialogue with Contextualized Stance Control](https://mlsanity.com/paper/2301.10368/) (2023)
- [HarmPot: An Annotation Framework for Evaluating Offline Harm Potential of Social Media Text](https://mlsanity.com/paper/2403.11108/) (2024)
- [A Systematic Survey of Natural Language Processing for the Greek Language](https://mlsanity.com/paper/2407.09861/) (2024)
- [Neural Models for Offensive Language Detection](https://mlsanity.com/paper/2106.14609/) (2021)
- [From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation](https://mlsanity.com/paper/2608.25605/) (2026)

## Abstract (arXiv)

This paper describes Galileo's performance in SemEval-2020 Task 12 on detecting and categorizing offensive language in social media. For Offensive Language Identification, we proposed a multi-lingual method using Pre-trained Language Models, ERNIE and XLM-R. For offensive language categorization, we proposed a knowledge distillation method trained on soft labels generated by several supervised models. Our team participated in all three sub-tasks. In Sub-task A - Offensive Language Identification, we ranked first in terms of average F1 scores in all languages. We are also the only team which ranked among the top three across all languages. We also took the first place in Sub-task B - Automatic Categorization of Offense Types and Sub-task C - Offence Target Identification.