mlsanityData to 14 Oct 2026

Unveiling the Implicit Toxicity in Large Language Models

arXiv 2311.17391 ↗First posted 29 Nov 2023cs.CLcited by 51

Not on the map. Why?

Cited by 27 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The open-endedness of large language models (LLMs) combined with their impressive capabilities may lead to new safety issues when being exploited for malicious use. While recent studies primarily focus on probing toxic outputs that can be easily detected with existing toxicity classifiers, we show that LLMs can generate diverse implicit toxic outputs that are exceptionally difficult to detect via simply zero-shot prompting. Moreover, we propose a reinforcement learning (RL) based attacking method to further induce the implicit toxicity in LLMs. Specifically, we optimize the language model with a reward that prefers implicit toxic outputs to explicit toxic and non-toxic ones. Experiments on five widely-adopted toxicity classifiers demonstrate that the attack success rate can be significantly improved through RL fine-tuning. For instance, the RL-finetuned LLaMA-13B model achieves an attack success rate of 90.04% on BAD and 62.85% on Davinci003. Our findings suggest that LLMs pose a significant threat in generating undetectable implicit toxic outputs. We further show that fine-tuning toxicity classifiers on the annotated examples from our attacking method can effectively enhance their ability to detect LLM-generated implicit toxic language. The code is publicly available at https://github.com/thu-coai/Implicit-Toxicity.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

31 of 46 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  8. 2020Learning to summarize from human feedbackcited by 2,588
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  10. 2020Extracting Training Data from Large Language Modelscited by 2,189
  11. 2019Language Models as Knowledge Bases?cited by 1,936
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
19 more
  1. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  2. 2022Red Teaming Language Models with Language Modelscited by 1,026
  3. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  4. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  5. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 736
  6. 2020Recipes for building an open-domain chatbotcited by 593
  7. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  8. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  9. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  10. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  11. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  12. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  13. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 217
  14. 2022Predictability and Surprise in Large Generative Modelscited by 210
  15. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  16. 2020Recipes for Safety in Open-domain Chatbotscited by 170
  17. 2021Can Machines Learn Morality? The Delphi Experimentcited by 141
  18. 2017Detecting Online Hate Speech Using Context Aware Modelscited by 72
  19. 2023Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimationcited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 17; 2025: 23; 2026: 11.

51 papers.

Sort:

By area:

  1. 2024Rethinking Machine Unlearning for Large Language Modelscited by 257
  2. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 212
  3. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  4. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  5. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 88
  6. 2024Safeguarding Large Language Models: A Surveycited by 78
  7. 2024From Pixels to Prose: A Large Dataset of Dense Image Captionscited by 54
  8. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMscited by 42
  9. 2025LLM Unlearning via Neural Activation Redirectioncited by 41
  10. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 37
  11. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 33
  12. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 26
38 more
  1. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  2. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearningcited by 23
  3. 2025From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoringcited by 22
  4. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuningcited by 21
  5. 2024Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Modelscited by 20
  6. 2024GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMscited by 12
  7. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMscited by 11
  8. 2025A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Biascited by 10
  9. 2024Statistical inference on black-box generative models in the data kernel perspective spacecited by 9
  10. 2025Improving LLM Unlearning Robustness via Random Perturbationscited by 9
  11. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 9
  12. 2025Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decodingcited by 9
  13. 2025IF-GUIDE: Influence Function-Guided Detoxification of LLMscited by 7
  14. 2026Eliciting Numerical Predictive Distributions of LLMs Without Autoregressioncited by 6
  15. 2024Verbosity neq Veracity: Demystify Verbosity Compensation Behavior of Large Language Modelscited by 5
  16. 2025Aligned Probing: Relating Toxic Behavior and Model Internalscited by 5
  17. 2025The Erasure Illusion: Stress-Testing the Generalization of LLM Forgetting Evaluationcited by 5
  18. 2024Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMscited by 4
  19. 2025BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Modelscited by 4
  20. 2025Chain-of-Thought Driven Adversarial Scenario Extrapolation for Robust Language Modelscited by 4
  21. 2025RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaborationcited by 4
  22. 2024ImpScore: A Learnable Metric For Quantifying The Implicitness Level of Sentencecited by 3
  23. 2025Fight Fire with Fire: Defending Against Malicious RL Fine-Tuning via Reward Neutralizationcited by 3
  24. 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directionscited by 3
  25. 2024The Superalignment of Superhuman Intelligence with Large Language Modelscited by 2
  26. 2025LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Datacited by 2
  27. 2026Mitigating Memorization Where It Happenscited by 2
  28. 2026Distinguishable Deletion: Unifying Knowledge Erasure and Refusal for Large Language Model Unlearningcited by 2
  29. 2026Detoxifying LLMs via Representation Erasure-Based Preference Optimizationcited by 1
  30. 2025Thematic and Task-Based Categorization of K-12 GenAI Usages with Hierarchical Topic Modelingcited by 0
  31. 2025UpSafe^circC: Upcycling for Controllable Safety in Large Language Modelscited by 0
  32. 2025Quantifying Document Impact in RAG-LLMscited by 0
  33. 2026Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forwardcited by 0
  34. 2026Provable Last-Iterate Convergence for Multi-Objective Safe LLM Alignment via Optimistic Primal-Dualcited by 0
  35. 2026Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Studycited by 0
  36. 2026ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Modelscited by 0
  37. 2026AERIC: Anticipatory Hidden-State Monitoring for Implicit Harmful Dialoguecited by 0
  38. 2026Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix itcited by 0

50 most cited of 51 All 51 (JSON)