# Ruddit: Norms of Offensiveness for English Reddit Comments

arXiv: https://arxiv.org/abs/2106.05664 · HTML: https://mlsanity.com/paper/2106.05664/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 69 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020)
- [Reducing Gender Bias in Abusive Language Detection](https://mlsanity.com/paper/1808.07231/) (2018)
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation](https://mlsanity.com/paper/1712.01765/) (2017)
- [Detecting Online Hate Speech Using Context Aware Models](https://mlsanity.com/paper/1710.07395/) (2017)
- [Tackling Online Abuse: A Survey of Automated Abuse Detection Methods](https://mlsanity.com/paper/1908.06024/) (2019)
- [Emotion Intensities in Tweets](https://mlsanity.com/paper/1708.03696/) (2017)
- [Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best-Worst Scaling](https://mlsanity.com/paper/1712.01741/) (2017)
- [Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter](https://mlsanity.com/paper/1811.02906/) (2018)
- [Towards Ethics by Design in Online Abusive Content Detection](https://mlsanity.com/paper/2010.14952/) (2020)

## Built on by (most cited first, 27 of 27)

- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs](https://mlsanity.com/paper/2406.18495/) (2024)
- [GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis](https://mlsanity.com/paper/2402.13494/) (2024)
- [Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts](https://mlsanity.com/paper/2108.11830/) (2021)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models](https://mlsanity.com/paper/2410.01524/) (2024)
- [A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations](https://mlsanity.com/paper/2505.14106/) (2025)
- [''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Text](https://mlsanity.com/paper/2310.17428/) (2023)
- [A Benchmark for Understanding Dialogue Safety in Mental Health Support](https://mlsanity.com/paper/2307.16457/) (2023)
- [Which Demographics do LLMs Default to During Annotation?](https://mlsanity.com/paper/2410.08820/) (2024)
- [The Unseen Targets of Hate -- A Systematic Review of Hateful Communication Datasets](https://mlsanity.com/paper/2405.08562/) (2024)
- [Evaluating how LLM annotations represent diverse views on contentious topics](https://mlsanity.com/paper/2503.23243/) (2025)
- [CRUSH: Contextually Regularized and User anchored Self-supervised Hate speech Detection](https://mlsanity.com/paper/2204.06389/) (2022)
- [On the rise of fear speech in online social media](https://mlsanity.com/paper/2303.10311/) (2023)
- [Which one is more toxic? Findings from Jigsaw Rate Severity of Toxic Comments](https://mlsanity.com/paper/2206.13284/) (2022)
- [QGuard:Question-based Zero-shot Guard for Multi-modal LLM Safety](https://mlsanity.com/paper/2506.12299/) (2025)
- [WLV-RIT at GermEval 2021: Multitask Learning with Transformers to Detect Toxic, Engaging, and Fact-Claiming Comments](https://mlsanity.com/paper/2108.00057/) (2021)
- ["You are an expert annotator": Automatic Best-Worst-Scaling Annotations for Emotion Intensity Modeling](https://mlsanity.com/paper/2403.17612/) (2024)
- [SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models](https://mlsanity.com/paper/2502.12464/) (2025)
- [Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems](https://mlsanity.com/paper/2510.13351/) (2025)
- [Analyzing the Intensity of Complaints on Social Media](https://mlsanity.com/paper/2204.09366/) (2022)
- [Style Locality for Controllable Generation with kNN Language Models](https://mlsanity.com/paper/2311.00475/) (2023)
- [Ultra Low-Cost Two-Stage Multimodal System for Non-Normative Behavior Detection](https://mlsanity.com/paper/2403.16151/) (2024)
- [PoPreRo: A New Dataset for Popularity Prediction of Romanian Reddit Posts](https://mlsanity.com/paper/2407.04541/) (2024)
- [Cross-Lingual Probing and Community-Grounded Analysis of Gender Bias in Low-Resource Bengali](https://mlsanity.com/paper/2601.17764/) (2026)
- [GASTON: Graph-Aware Social Transformer for Online Networks](https://mlsanity.com/paper/2602.02524/) (2026)
- [Demographic Prompting at Scale: When More Attributes Hurt LLM--Human Agreement](https://mlsanity.com/paper/2607.10590/) (2026)

## Abstract (arXiv)

On social media platforms, hateful and offensive language negatively impact the mental well-being of users and the participation of people from diverse backgrounds. Automatic methods to detect offensive language have largely relied on datasets with categorical labels. However, comments can vary in their degree of offensiveness. We create the first dataset of English language Reddit comments that has fine-grained, real-valued scores between -1 (maximally supportive) and 1 (maximally offensive). The dataset was annotated using Best--Worst Scaling, a form of comparative annotation that has been shown to alleviate known biases of using rating scales. We show that the method produces highly reliable offensiveness scores. Finally, we evaluate the ability of widely-used neural models to predict offensiveness scores on this new dataset.