# On The Robustness of Offensive Language Classifiers

arXiv: https://arxiv.org/abs/2203.11331 · HTML: https://mlsanity.com/paper/2203.11331/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems](https://mlsanity.com/paper/1903.11508/) (2019)
- [All You Need is "Love": Evading Hate-speech Detection](https://mlsanity.com/paper/1808.09115/) (2018)
- [Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach](https://mlsanity.com/paper/1805.05181/) (2018)
- [Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platforms](https://mlsanity.com/paper/1801.06482/) (2018)
- [Identifying Offensive Posts and Targeted Offense from Twitter](https://mlsanity.com/paper/1904.09072/) (2019)

## Built on by (most cited first, 8 of 8)

- [Cross-Cultural Transfer Learning for Chinese Offensive Language Detection](https://mlsanity.com/paper/2303.17927/) (2023)
- [ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models](https://mlsanity.com/paper/2310.09624/) (2023)
- [A Note on Bias to Complete](https://mlsanity.com/paper/2402.11710/) (2024)
- [VertAttack: Taking advantage of Text Classifiers' horizontal vision](https://mlsanity.com/paper/2404.08538/) (2024)
- [BinarySelect to Improve Accessibility of Black-Box Attack Research](https://mlsanity.com/paper/2412.10617/) (2024)
- [Relation Extraction or Pattern Matching? Unravelling the Generalisation Limits of Language Models for Biographical RE](https://mlsanity.com/paper/2505.12533/) (2025)
- [NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?](https://mlsanity.com/paper/2211.04364/) (2022)
- [Adversarial Testing for Visual Grounding via Image-Aware Property Reduction](https://mlsanity.com/paper/2403.01118/) (2024)

## Abstract (arXiv)

Social media platforms are deploying machine learning based offensive language classification systems to combat hateful, racist, and other forms of offensive speech at scale. However, despite their real-world deployment, we do not yet comprehensively understand the extent to which offensive language classifiers are robust against adversarial attacks. Prior work in this space is limited to studying robustness of offensive language classifiers against primitive attacks such as misspellings and extraneous spaces. To address this gap, we systematically analyze the robustness of state-of-the-art offensive language classifiers against more crafty adversarial attacks that leverage greedy- and attention-based word selection and context-aware embeddings for word replacement. Our results on multiple datasets show that these crafty adversarial attacks can degrade the accuracy of offensive language classifiers by more than 50% while also being able to preserve the readability and meaning of the modified text.