Analyzing Toxicity in Deep Conversations: A Reddit Case Study
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Online social media has become increasingly popular in recent years due to its ease of access and ability to connect with others. One of social media's main draws is its anonymity, allowing users to share their thoughts and opinions without fear of judgment or retribution. This anonymity has also made social media prone to harmful content, which requires moderation to ensure responsible and productive use. Several methods using artificial intelligence have been employed to detect harmful content. However, conversation and contextual analysis of hate speech are still understudied. Most promising works only analyze a single text at a time rather than the conversation supporting it. In this work, we employ a tree-based approach to understand how users behave concerning toxicity in public conversation settings. To this end, we collect both the posts and the comment sections of the top 100 posts from 8 Reddit communities that allow profanity, totaling over 1 million responses. We find that toxic comments increase the likelihood of subsequent toxic comments being produced in online conversations. Our analysis also shows that immediate context plays a vital role in shaping a response rather than the original post. We also study the effect of consensual profanity and observe overlapping similarities with non-consensual profanity in terms of user behavior and patterns.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 18 of its 38 references that are in the map. First posted on arXiv 11 Apr 2024.
Builds on
18 of 38 references.
References · 18
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
- 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
- 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
- 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
- 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 217
- 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 216
- 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 215
- 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
- 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
- 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
- 2020Demoting Racial Bias in Hate Speech Detectioncited by 62
6 more
- 2018Sarcasm Analysis using Conversation Contextcited by 17
- 2023CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Networkcited by 12
- 2022Public Wisdom Matters! Discourse-Aware Hyperbolic Fourier Co-Attention for Social-Text Classificationcited by 11
- 2021Successful New-entry Prediction for Multi-Party Online Conversations via Latent Topics and Discourse Modelingcited by 2
- 2022Twitter Users' Behavioral Response to Toxic Repliescited by 2
- 2023Comparing Toxicity Across Social Media Platforms for COVID-19 Discoursecited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 2.
3 papers.