Analyzing Norm Violations in Live-Stream Chat
Not on the map. Why?
Abstract (arXiv)
Toxic language, such as hate speech, can deter users from participating in online communities and enjoying popular platforms. Previous approaches to detecting toxic language and norm violations have been primarily concerned with conversations from online forums and social media, such as Reddit and Twitter. These approaches are less effective when applied to conversations on live-streaming platforms, such as Twitch and YouTube Live, as each comment is only visible for a limited time and lacks a thread structure that establishes its relationship with other comments. In this work, we share the first NLP study dedicated to detecting norm violations in conversations on live-streaming platforms. We define norm violation categories in live-stream chats and annotate 4,583 moderated comments from Twitch. We articulate several facets of live-stream data that differ from other forums, and demonstrate that existing models perform poorly in this setting. By conducting a user study, we identify the informational context humans use in live-stream moderation, and train models leveraging context to identify norm violations. Our results show that appropriate contextual information can boost moderation performance by 35%.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 18 May 2023.
Builds on
25 of 46 references.
References · 25
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 736
- 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
- 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 407
- 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
- 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
- 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
- 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 270
- 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 264
- 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 217
- 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 185
- 2019Multilingual and Multi-Aspect Hate Speech Analysiscited by 133
13 more
- 2020Toxicity Detection: Does Context Really Matter?cited by 93
- 2020Contextualizing Hate Speech Classifiers with Post-hoc Explanationcited by 84
- 2018Community Interaction and Conflict on the Webcited by 74
- 2016Analyzing the Targets of Hate in Online Social Mediacited by 49
- 2019A Just and Comprehensive Strategy for Using NLP to Address Online Abusecited by 49
- 2019Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Studycited by 47
- 2021Abuse is Contextual, What about NLP? The Role of Context in Abusive Language Annotation and Detectioncited by 19
- 2021Detecting Community Sensitive Norm Violations in Online Conversationscited by 18
- 2022XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Modelscited by 8
- 2017Surfacing contextual hate speech words within social mediacited by 6
- 2018Extracting Inter-community Conflicts in Redditcited by 6
- 2020Norm violation in online communities -- A study of Stack Overflow commentscited by 5
- 2021Abusive Language Detection in Heterogeneous Contexts: Dataset Collection and the Role of Supervised Attentioncited by 5
Built on by
Citing papers per year: 2024: 2; 2025: 0; 2026: 1.
3 papers.