# A Federated Learning Approach to Privacy Preserving Offensive Language Identification

arXiv: https://arxiv.org/abs/2404.11470 · HTML: https://mlsanity.com/paper/2404.11470/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Fusing finetuned models for better pretraining](https://mlsanity.com/paper/2204.03044/) (2022)
- [Multilingual Offensive Language Identification with Cross-lingual Embeddings](https://mlsanity.com/paper/2010.05324/) (2020)
- [Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization](https://mlsanity.com/paper/2310.15080/) (2023)
- [SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification](https://mlsanity.com/paper/2004.14454/) (2020)
- [Detecting Hate Speech in Social Media](https://mlsanity.com/paper/1712.06427/) (2017)
- [Federated Learning of N-gram Language Models](https://mlsanity.com/paper/1910.03432/) (2019)
- [Experiments of Federated Learning for COVID-19 Chest X-ray Images](https://mlsanity.com/paper/2007.05592/) (2020)
- [MUDES: Multilingual Detection of Offensive Spans](https://mlsanity.com/paper/2102.09665/) (2021)
- [FBERT: A Neural Transformer for Identifying Offensive Content](https://mlsanity.com/paper/2109.05074/) (2021)
- [Arabic Offensive Language on Twitter: Analysis and Experiments](https://mlsanity.com/paper/2004.02192/) (2020)
- [Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi](https://mlsanity.com/paper/2109.03552/) (2021)
- [Challenges in Discriminating Profanity from Hate Speech](https://mlsanity.com/paper/1803.05495/) (2018)
- [Offensive Language Identification in Transliterated and Code-Mixed Bangla](https://mlsanity.com/paper/2311.15023/) (2023)
- [DeL-haTE: A Deep Learning Tunable Ensemble for Hate Speech Detection](https://mlsanity.com/paper/2011.01861/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Federated Approach to Few-Shot Hate Speech Detection for Marginalized Communities](https://mlsanity.com/paper/2412.04942/) (2024)
- [Improving Hate Speech Classification with Cross-Taxonomy Dataset Integration](https://mlsanity.com/paper/2503.05357/) (2025)
- [Evaluating Apple Intelligence's Writing Tools for Privacy Against Large Language Model-Based Inference Attacks: Insights from Early Datasets](https://mlsanity.com/paper/2506.03870/) (2025)
- [NLP Privacy Risk Identification in Social Media (NLP-PRISM): A Survey](https://mlsanity.com/paper/2602.15866/) (2026)

## Abstract (arXiv)

The spread of various forms of offensive speech online is an important concern in social media. While platforms have been investing heavily in ways of coping with this problem, the question of privacy remains largely unaddressed. Models trained to detect offensive language on social media are trained and/or fine-tuned using large amounts of data often stored in centralized servers. Since most social media data originates from end users, we propose a privacy preserving decentralized architecture for identifying offensive language online by introducing Federated Learning (FL) in the context of offensive language identification. FL is a decentralized architecture that allows multiple models to be trained locally without the need for data sharing hence preserving users' privacy. We propose a model fusion approach to perform FL. We trained multiple deep learning models on four publicly available English benchmark datasets (AHSD, HASOC, HateXplain, OLID) and evaluated their performance in detail. We also present initial cross-lingual experiments in English and Spanish. We show that the proposed model fusion approach outperforms baselines in all the datasets while preserving privacy.