Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions
Not on the map. Why?
Abstract (arXiv)
The evolution of digital communication systems and the designs of online platforms have inadvertently facilitated the subconscious propagation of toxic behavior. Giving rise to reactive responses to toxic behavior. Toxicity in online content and Artificial Intelligence Systems has become a serious challenge to individual and collective well-being around the world. It is more detrimental to society than we realize. Toxicity, expressed in language, image, and video, can be interpreted in various ways depending on the context of usage. Therefore, a comprehensive taxonomy is crucial to detect and mitigate toxicity in online content, Artificial Intelligence systems, and/or Large Language Models in a proactive manner. A comprehensive understanding of toxicity is likely to facilitate the design of practical solutions for toxicity detection and mitigation. The classification in published literature has focused on only a limited number of aspects of this very complex issue, with a pattern of reactive strategies in response to toxicity. This survey attempts to generate a comprehensive taxonomy of toxicity from various perspectives. It presents a holistic approach to explain the toxicity by understanding the context and environment that society is facing in the Artificial Intelligence era. This survey summarizes the toxicity-related datasets and research on toxicity detection and mitigation for Large Language Models, social media platforms, and other online platforms, detailing their attributes in textual mode, focused on the English language. Finally, we suggest the research gaps in toxicity mitigation based on datasets, mitigation strategies, Large Language Models, adaptability, explainability, and evaluation.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 112 of its 243 references that are in the map. First posted on arXiv 29 Sep 2025.
Builds on
112 of 243 references.
References · 112
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
- 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
- 2023Bias and Fairness in Large Language Models: A Surveycited by 639
38 more
- 2019Defending Against Neural Fake Newscited by 559
- 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
- 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
- 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 407
- 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
- 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
- 2018FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Mediacited by 285
- 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
- 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 234
- 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 221
- 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 217
- 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 216
- 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
- 2023Personality Traits in Large Language Modelscited by 169
- 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
- 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
- 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 148
- 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 136
- 2023Hallucinations in Large Multilingual Translation Modelscited by 129
- 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 106
- 2023The science of fake newscited by 97
- 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
- 2021Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speechcited by 82
- 2023Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detectioncited by 72
- 2023Can ChatGPT Assess Human Personalities? A General Evaluation Frameworkcited by 70
- 2022Handling Bias in Toxic Speech Detection: A Surveycited by 61
- 2018Challenges for Toxic Comment Classification: An In-Depth Error Analysiscited by 56
- 2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
- 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 56
- 2023Self-Guard: Empower the LLM to Safeguard Itselfcited by 54
- 2023Unveiling the Implicit Toxicity in Large Language Modelscited by 51
- 2024Measuring Implicit Bias in Explicitly Unbiased Large Language Modelscited by 51
- 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
- 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
- 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
- 2020Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspectivecited by 43
50 most cited of 112 All 112 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Modelscited by 1
- 2026Beyond Inefficiency: Systemic Costs of Incivility in Multi-Agent Monte Carlo Simulationscited by 0
- 2026Calling the Bluff: Detecting Ever-Shifting Harmful Chat Dialogue via Ordered Reasoning Chain Regularizationcited by 0
- 2026Calling the Bluff: Detecting Ever-Shifting Harmful Chat Dialogue via Ordered Reasoning Chain Regularization9 Aug 2026
- 2026Beyond Inefficiency: Systemic Costs of Incivility in Multi-Agent Monte Carlo Simulations12 May 2026
- 2026Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models18 Feb 2026