mlsanityData to 7 Oct 2026

On The Robustness of Offensive Language Classifiers

arXiv 2203.11331 ↗First posted 21 Mar 2022cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Social media platforms are deploying machine learning based offensive language classification systems to combat hateful, racist, and other forms of offensive speech at scale. However, despite their real-world deployment, we do not yet comprehensively understand the extent to which offensive language classifiers are robust against adversarial attacks. Prior work in this space is limited to studying robustness of offensive language classifiers against primitive attacks such as misspellings and extraneous spaces. To address this gap, we systematically analyze the robustness of state-of-the-art offensive language classifiers against more crafty adversarial attacks that leverage greedy- and attention-based word selection and context-aware embeddings for word replacement. Our results on multiple datasets show that these crafty adversarial attacks can degrade the accuracy of offensive language classifiers by more than 50% while also being able to preserve the readability and meaning of the modified text.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 16 of its 39 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

16 of 39 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2016Adversarial Machine Learning at Scalecited by 934
  4. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  5. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  6. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  7. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  8. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 268
  9. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
  10. 2017Generating Natural Adversarial Examplescited by 182
  11. 2020SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)cited by 138
  12. 2019Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systemscited by 99
4 more
  1. 2018All You Need is "Love": Evading Hate-speech Detectioncited by 56
  2. 2018Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approachcited by 55
  3. 2018Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platformscited by 37
  4. 2019Identifying Offensive Posts and Targeted Offense from Twittercited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 4; 2025: 1.

8 papers.

  1. 2023Cross-Cultural Transfer Learning for Chinese Offensive Language Detectioncited by 21
  2. 2023ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Modelscited by 12
  3. 2024A Note on Bias to Completecited by 1
  4. 2024VertAttack: Taking advantage of Text Classifiers' horizontal visioncited by 1
  5. 2024BinarySelect to Improve Accessibility of Black-Box Attack Researchcited by 1
  6. 2025Relation Extraction or Pattern Matching? Unravelling the Generalisation Limits of Language Models for Biographical REcited by 1
  7. 2022NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?cited by 0
  8. 2024Adversarial Testing for Visual Grounding via Image-Aware Property Reductioncited by 0