mlsanityData to 7 Oct 2026

Towards Building a Robust Toxicity Predictor

arXiv 2404.08690 ↗First posted 9 Apr 2024cs.CL cs.AI cs.CRcited by 14

Not on the map. Why?

Abstract (arXiv)

Recent NLP literature pays little attention to the robustness of toxicity language predictors, while these systems are most likely to be used in adversarial contexts. This paper presents a novel adversarial attack, textttToxicTrap, introducing small word-level perturbations to fool SOTA text classifiers to predict toxic text samples as benign. ToxicTrap exploits greedy based search strategies to enable fast and effective generation of toxic adversarial examples. Two novel goal function designs allow ToxicTrap to identify weaknesses in both multiclass and multilabel toxic language detectors. Our empirical results show that SOTA toxicity text classifiers are indeed vulnerable to the proposed attacks, attaining over 98% attack success rates in multilabel cases. We also show how a vanilla adversarial training and its improved version can help increase robustness of a toxicity detector even against unseen attacks.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

22 of 41 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  8. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  9. 2019On Evaluating Adversarial Robustnesscited by 442
  10. 2018Generating Natural Language Adversarial Examplescited by 422
  11. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  12. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
10 more
  1. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  2. 2020SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)cited by 138
  3. 2017Deceiving Google's Perspective API Built for Detecting Toxic Commentscited by 99
  4. 2018Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examplescited by 93
  5. 2020Reevaluating Adversarial Examples in Natural Languagecited by 83
  6. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  7. 2020TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLPcited by 45
  8. 2019Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Surveycited by 10
  9. 2021ToxCCIn: Toxic Content Classification with Interpretabilitycited by 10
  10. 2021Cisco at SemEval-2021 Task 5: What's Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Commentscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 8; 2026: 1.

14 papers.

Sort:

By area:

  1. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  2. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  3. 2024Enhancing LLM-based Hatred and Toxicity Detection with Meta-Toxic Knowledge Graphcited by 4
  4. 2024DoPAMine: Domain-specific Pre-training Adaptation from seed-guided data Miningcited by 3
  5. 2025Automating Prompt Leakage Attacks on Large Language Models Using Agentic Approachcited by 3
  6. 2025Breaking mBad! Supervised Fine-tuning for Cross-Lingual Detoxificationcited by 3
  7. 2025Trick or Neat: Adversarial Ambiguity and Language Model Evaluationcited by 3
  8. 2025ToxiFrench: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detectioncited by 3
  9. 2024TaeBench: Improving Quality of Toxic Adversarial Examplescited by 2
  10. 2025Language Models That Walk the Talk: A Framework for Formal Fairness Certificatescited by 0
  11. 2025Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Contentcited by 0
  12. 2025SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Modelscited by 0
2 more
  1. 2025Keep the Lights On, Keep the Lengths in Check: Plug-In Adversarial Detection for Time-Series LLMs in Energy Forecastingcited by 0
  2. 2026ContiGuard: A Framework for Continual Toxicity Detection Against Evolving Evasive Perturbationscited by 0