mlsanityData to 7 Oct 2026

Deceiving Google's Perspective API Built for Detecting Toxic Comments

arXiv 1702.08138 ↗First posted 27 Feb 2017cs.LG cs.CY cs.SIcited by 99

Not on the map. Why?

Abstract (arXiv)

Social media platforms provide an environment where people can freely engage in discussions. Unfortunately, they also enable several problems, such as online harassment. Recently, Google and Jigsaw started a project called Perspective, which uses machine learning to automatically detect toxic language. A demonstration website has been also launched, which allows anyone to type a phrase in the interface and instantaneously see the toxicity score [1]. In this paper, we propose an attack on the Perspective toxic detection system based on the adversarial examples. We show that an adversary can subtly modify a highly toxic phrase in a way that the system assigns significantly lower toxicity score to it. We apply the attack on the sample phrases provided in the Perspective website and show that we can consistently reduce the toxicity scores to the level of the non-toxic phrases. The existence of such adversarial examples is very harmful for toxic detection systems and seriously undermines their usability.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 20; 2023: 24; 2024: 17; 2025: 11; 2026: 7.

99 papers.

Sort:

By area:

  1. 2022Red Teaming Language Models with Language Modelscited by 983
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  3. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  4. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  5. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  6. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  7. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  8. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  9. 2021Semantic Communications: Principles and Challengescited by 90
  10. 2022AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversationcited by 79
  11. 2024Safeguarding Large Language Models: A Surveycited by 73
  12. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
38 more
  1. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  2. 2023"I'm fully who I am": Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generationcited by 56
  3. 2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
  4. 2022Critic-Guided Decoding for Controlled Text Generationcited by 45
  5. 2023Evaluating GPT-3 Generated Explanations for Hateful Content Moderationcited by 44
  6. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  7. 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
  8. 2024Language Models for Text Classification: Is In-Context Learning Enough?cited by 39
  9. 2024Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characterscited by 39
  10. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
  11. 2023You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Contentcited by 37
  12. 2023Flames: Benchmarking Value Alignment of LLMs in Chinesecited by 35
  13. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  14. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
  15. 2024Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Modelscited by 31
  16. 2023Adversarial Illusions in Multi-Modal Embeddingscited by 28
  17. 2022Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processingcited by 23
  18. 2022Certified Robustness Against Natural Language Attacks by Causal Interventioncited by 23
  19. 2023Intuitive or Dependent? Investigating LLMs' Behavior Style to Conflicting Promptscited by 23
  20. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 22
  21. 2023(Why) Is My Prompt Getting Worse? Rethinking Regression Testing for Evolving LLM APIscited by 21
  22. 2021Efficient Combinatorial Optimization for Word-level Adversarial Textual Attackcited by 17
  23. 2023Evolving Diverse Red-team Language Models in Multi-round Multi-agent Gamescited by 17
  24. 2023''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Textcited by 15
  25. 2021Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classificationcited by 14
  26. 2024Towards Building a Robust Toxicity Predictorcited by 14
  27. 2025ArithmAttack: Evaluating Robustness of LLMs to Noisy Context in Math Problem Solvingcited by 13
  28. 2021Fall of Giants: How popular text-based MLaaS fall against a simple evasion attackcited by 12
  29. 2021Interpretable Multi-Modal Hate Speech Detectioncited by 10
  30. 2025HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaignscited by 10
  31. 2021Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnosticscited by 9
  32. 2022Robust Natural Language Processing: Recent Advances, Challenges, and Future Directionscited by 9
  33. 2023Less is More: Understanding Word-level Textual Adversarial Attack via n-gram Frequency Descendcited by 9
  34. 2022Automated Identification of Toxic Code Reviews Using ToxiCRcited by 8
  35. 2025MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attackscited by 8
  36. 2021Measuring Model Fairness under Noisy Covariates: A Theoretical Perspectivecited by 7
  37. 2021RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualizationcited by 6
  38. 2023Critical Perspectives: A Benchmark Revealing Pitfalls in PerspectiveAPIcited by 6

50 most cited of 99 All 99 (JSON)