# Detecting Community Sensitive Norm Violations in Online Conversations

arXiv: https://arxiv.org/abs/2110.04419 · HTML: https://mlsanity.com/paper/2110.04419/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 9 Oct 2021.

## Builds on (most cited first, 14 of 14)

- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation](https://mlsanity.com/paper/1911.03842/) (2019)
- [Toxicity Detection: Does Context Really Matter?](https://mlsanity.com/paper/2006.00998/) (2020)
- [Contextualizing Hate Speech Classifiers with Post-hoc Explanation](https://mlsanity.com/paper/2005.02439/) (2020)
- [Conversations Gone Awry: Detecting Early Signs of Conversational Failure](https://mlsanity.com/paper/1805.05345/) (2018)
- [A Just and Comprehensive Strategy for Using NLP to Address Online Abuse](https://mlsanity.com/paper/1906.01738/) (2019)
- [Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop](https://mlsanity.com/paper/1909.01362/) (2019)
- [Like trainer, like bot? Inheritance of bias in algorithmic content moderation](https://mlsanity.com/paper/1707.01477/) (2017)
- [Unsupervised Discovery of Implicit Gender Bias](https://mlsanity.com/paper/2004.08361/) (2020)
- [Something's Brewing! Early Prediction of Controversy-causing Posts from Discussion Features](https://mlsanity.com/paper/1904.07372/) (2019)
- [Surviving an "Eternal September" - How an Online Community Managed a Surge of Newcomers](https://mlsanity.com/paper/1605.08841/) (2016)

## Built on by (most cited first, 18 of 18)

- [Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark](https://mlsanity.com/paper/2305.14938/) (2023)
- [MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance](https://mlsanity.com/paper/2505.14483/) (2025)
- [Can Language Model Moderators Improve the Health of Online Discourse?](https://mlsanity.com/paper/2311.10781/) (2023)
- [ValueScope: Unveiling Implicit Norms and Values via Return Potential Model of Social Interactions](https://mlsanity.com/paper/2407.02472/) (2024)
- [CPL-NoViD: Context-Aware Prompt-based Learning for Norm Violation Detection in Online Communities](https://mlsanity.com/paper/2305.09846/) (2023)
- [Your spouse needs professional help: Determining the Contextual Appropriateness of Messages through Modeling Social Relationships](https://mlsanity.com/paper/2307.02763/) (2023)
- [Leveraging Prompt-Based Large Language Models: Predicting Pandemic Health Decisions and Outcomes Through Social Media Language](https://mlsanity.com/paper/2403.00994/) (2024)
- [Argument Quality Assessment in the Age of Instruction-Following Large Language Models](https://mlsanity.com/paper/2403.16084/) (2024)
- [Anger Breeds Controversy: Analyzing Controversy and Emotions on Reddit](https://mlsanity.com/paper/2212.00339/) (2022)
- [Analyzing Norm Violations in Live-Stream Chat](https://mlsanity.com/paper/2305.10731/) (2023)
- [Promoting Constructive Deliberation: Reframing for Receptiveness](https://mlsanity.com/paper/2405.15067/) (2024)
- [WHoW: A Cross-domain Approach for Analysing Conversation Moderation](https://mlsanity.com/paper/2410.15551/) (2024)
- [The Gray Area: Characterizing Moderator Disagreement on Reddit](https://mlsanity.com/paper/2601.01620/) (2026)
- [Decoding the Rule Book: Extracting Hidden Moderation Criteria from Reddit Communities](https://mlsanity.com/paper/2509.02926/) (2025)
- [Asking For It: Question-Answering for Predicting Rule Infractions in Online Content Moderation](https://mlsanity.com/paper/2510.06350/) (2025)
- [SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations](https://mlsanity.com/paper/2511.07405/) (2025)
- [GASTON: Graph-Aware Social Transformer for Online Networks](https://mlsanity.com/paper/2602.02524/) (2026)
- [PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media](https://mlsanity.com/paper/2605.17187/) (2026)

## Abstract (arXiv)

Online platforms and communities establish their own norms that govern what behavior is acceptable within the community. Substantial effort in NLP has focused on identifying unacceptable behaviors and, recently, on forecasting them before they occur. However, these efforts have largely focused on toxicity as the sole form of community norm violation. Such focus has overlooked the much larger set of rules that moderators enforce. Here, we introduce a new dataset focusing on a more complete spectrum of community norms and their violations in the local conversational and global community contexts. We introduce a series of models that use this data to develop context- and community-sensitive norm violation detection, showing that these changes give high performance.