mlsanityData to 7 Oct 2026

Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approach

arXiv 2512.17367 ↗First posted 19 Dec 2025cs.LGcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Social media platforms are plagued by harmful content such as hate speech, misinformation, and extremist rhetoric. Machine learning (ML) models are widely adopted to detect such content; however, they remain highly vulnerable to adversarial attacks, wherein malicious users subtly modify text to evade detection. Enhancing adversarial robustness is therefore essential, requiring detectors that can defend against diverse attacks (generalizability) while maintaining high overall accuracy. However, simultaneously achieving both optimal generalizability and accuracy is challenging. Following the computational design science paradigm, this study takes a sequential approach that first proposes a novel framework (Large Language Model-based Sample Generation and Aggregation, LLM-SGA) by identifying the key invariances of textual adversarial attacks and leveraging them to ensure that a detector instantiated within the framework has strong generalizability. Second, we instantiate our detector (Adversarially Robust Harmful Online Content Detector, ARHOCD) with three novel design components to improve detection accuracy: (1) an ensemble of multiple base detectors that exploits their complementary strengths; (2) a novel weight assignment method that dynamically adjusts weights based on each sample's predictability and each base detector's capability, with weights initialized using domain knowledge and updated via Bayesian inference; and (3) a novel adversarial training strategy that iteratively optimizes both the base detectors and the weight assignor. We addressed several limitations of existing adversarial robustness enhancement research and empirically evaluated ARHOCD across three datasets spanning hate speech, rumor, and extremist content. Results show that ARHOCD offers strong generalizability and improves detection accuracy under adversarial conditions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 36 of its 64 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

36 of 64 references.

References · 36

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2019Invariant Risk Minimizationcited by 1,832
  5. 2016Adversarial Machine Learning at Scalecited by 934
  6. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  7. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  8. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  9. 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
  10. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  11. 2016Crafting Adversarial Input Sequences for Recurrent Neural Networkscited by 102
  12. 2020Reevaluating Adversarial Examples in Natural Languagecited by 83
24 more
  1. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  2. 2020SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutionscited by 72
  3. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  4. 2020ETHOS: an Online Hate Speech Detection Datasetcited by 66
  5. 2018Adversarial Examples: Opportunities and Challengescited by 62
  6. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57
  7. 2020Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examplescited by 54
  8. 2023Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attackscited by 32
  9. 2020EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attackscited by 28
  10. 2024Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attackcited by 27
  11. 2022Textual Manifold-based Defense Against Natural Language Adversarial Examplescited by 19
  12. 2023Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularizationcited by 19
  13. 2022Improving the Adversarial Robustness of NLP Models by Information Bottleneckcited by 17
  14. 2022Text Adversarial Purification as Defense against Adversarial Attackscited by 15
  15. 2022Robust Textual Embedding against Word-level Adversarial Attackscited by 12
  16. 2022Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretationcited by 12
  17. 2024Attacking Misinformation Detection Using Adversarial Examples Generated by Language Modelscited by 10
  18. 2024Adversarial Text Purification: A Large Language Model Approach for Defensecited by 8
  19. 2024Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-offcited by 5
  20. 2022Preserving Semantics in Textual Adversarial Attackscited by 4
  21. 2023Exploiting Explainability to Design Adversarial Attacks and Evaluate Attack Resilience in Hate-Speech Detection Modelscited by 2
  22. 2023Dynamic ensemble selection based on Deep Neural Network Uncertainty Estimation for Adversarial Robustnesscited by 2
  23. 2024Deep Learning Approaches for Detecting Adversarial Cyberbullying and Hate Speech in Social Networkscited by 2
  24. 2024MEAT: Median-Ensemble Adversarial Training for Improving Robustness and Generalizationcited by 1

Built on by

Not cited yet.