# Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models

arXiv: https://arxiv.org/abs/2210.13029 · HTML: https://mlsanity.com/paper/2210.13029/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020)
- [Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks](https://mlsanity.com/paper/1811.01088/) (2018)
- [SemEval-2017 Task 4: Sentiment Analysis in Twitter](https://mlsanity.com/paper/1912.00741/) (2019)
- [Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?](https://mlsanity.com/paper/2005.00628/) (2020)
- [Contextualizing Hate Speech Classifiers with Post-hoc Explanation](https://mlsanity.com/paper/2005.02439/) (2020)
- [Multilingual Offensive Language Identification with Cross-lingual Embeddings](https://mlsanity.com/paper/2010.05324/) (2020)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models](https://mlsanity.com/paper/2206.09917/) (2022)
- [Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP](https://mlsanity.com/paper/2005.14672/) (2020)
- [From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding](https://mlsanity.com/paper/2105.07316/) (2021)
- [English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too](https://mlsanity.com/paper/2005.13013/) (2020)
- [Models In a Spelling Bee: Language Models Implicitly Learn the Character Composition of Tokens](https://mlsanity.com/paper/2108.11193/) (2021)
- [Cross-lingual Zero- and Few-shot Hate Speech Detection Utilising Frozen Transformer Language Models and AXEL](https://mlsanity.com/paper/2004.13850/) (2020)
- [Zero-Resource Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/1911.09812/) (2019)
- [Treebanking User-Generated Content: a UD Based Overview of Guidelines, Corpora and Unified Recommendations](https://mlsanity.com/paper/2011.02063/) (2020)
- [Multi-Task Learning with Sentiment, Emotion, and Target Detection to Recognize Hate Speech and Offensive Language](https://mlsanity.com/paper/2109.10255/) (2021)
- [Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?](https://mlsanity.com/paper/2110.13658/) (2021)
- [Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis](https://mlsanity.com/paper/2201.07281/) (2022)
- [A cost-benefit analysis of cross-lingual transfer methods](https://mlsanity.com/paper/2105.06813/) (2021)
- [Bleaching Text: Abstract Features for Cross-lingual Gender Prediction](https://mlsanity.com/paper/1805.03122/) (2018)
- [Enhancing deep neural networks with morphological information](https://mlsanity.com/paper/2011.12432/) (2020)

## Built on by (most cited first, 10 of 10)

- [Cloaked Classifiers: Pseudonymization Strategies on Sensitive Classification Tasks](https://mlsanity.com/paper/2406.17875/) (2024)
- [Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection](https://mlsanity.com/paper/2412.11745/) (2024)
- [Common Ground, Diverse Roots: The Difficulty of Classifying Common Examples in Spanish Varieties](https://mlsanity.com/paper/2412.11750/) (2024)
- [Improving Dialectal Slot and Intent Detection with Auxiliary Tasks: A Multi-Dialectal Bavarian Case Study](https://mlsanity.com/paper/2501.03863/) (2025)
- [IYKYK: Using language models to decode extremist cryptolects](https://mlsanity.com/paper/2506.05635/) (2025)
- [Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challenges](https://mlsanity.com/paper/2401.09244/) (2024)
- [Add Noise, Tasks, or Layers? MaiNLP at the VarDial 2025 Shared Task on Norwegian Dialectal Slot and Intent Detection](https://mlsanity.com/paper/2501.03870/) (2025)
- [Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data](https://mlsanity.com/paper/2306.03722/) (2023)
- [Multi-lingual Functional Evaluation for Large Language Models](https://mlsanity.com/paper/2506.20793/) (2025)
- [Multilingual Hate Speech Detection and Counterspeech Generation: A Comprehensive Survey and Practical Guide](https://mlsanity.com/paper/2603.19279/) (2026)

## Abstract (arXiv)

Zero-shot cross-lingual transfer learning has been shown to be highly challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as in hate speech detection. In this paper, we highlight this limitation for hate speech detection in several domains and languages using strict experimental settings. Then, we propose to train on multilingual auxiliary tasks -- sentiment analysis, named entity recognition, and tasks relying on syntactic information -- to improve zero-shot transfer of hate speech detection models across languages. We show how hate speech detection models benefit from a cross-lingual knowledge proxy brought by auxiliary tasks fine-tuning and highlight these tasks' positive impact on bridging the hate speech linguistic and cultural gap between languages.