# IYKYK (But AI Doesn't): Automated Content Moderation Does Not Capture Communities' Heterogeneous Attitudes Towards Reclaimed Language

arXiv: https://arxiv.org/abs/2604.16654 · HTML: https://mlsanity.com/paper/2604.16654/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 96 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [Deliberative Alignment: Reasoning Enables Safer Language Models](https://mlsanity.com/paper/2412.16339/) (2024)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [NLPositionality: Characterizing Design Biases of Datasets and Models](https://mlsanity.com/paper/2306.01943/) (2023)
- [When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks](https://mlsanity.com/paper/2305.06626/) (2023)
- [Stop Measuring Calibration When Humans Disagree](https://mlsanity.com/paper/2210.16133/) (2022)
- [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation](https://mlsanity.com/paper/2205.00501/) (2022)
- [Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Lists](https://mlsanity.com/paper/2203.09192/) (2022)
- [Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation](https://mlsanity.com/paper/2112.04554/) (2021)
- [Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information](https://mlsanity.com/paper/2301.05036/) (2023)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis](https://mlsanity.com/paper/2308.16705/) (2023)
- [You Are What You Annotate: Towards Better Models through Annotator Representations](https://mlsanity.com/paper/2305.14663/) (2023)
- [The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels](https://mlsanity.com/paper/2405.05860/) (2024)
- [Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor](https://mlsanity.com/paper/2305.10374/) (2023)
- [Harmful Speech Detection by Language Models Exhibits Gender-Queer Dialect Bias](https://mlsanity.com/paper/2406.00020/) (2024)
- [Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization](https://mlsanity.com/paper/2404.00530/) (2024)
- [Subjective textitIsms? On the Danger of Conflating Hate and Offence in Abusive Language Detection](https://mlsanity.com/paper/2403.02268/) (2024)
- [Hidden behind the obvious: misleading keywords and implicitly abusive language on social media](https://mlsanity.com/paper/2205.01374/) (2022)
- [Do Users Want Platform Moderation or Individual Control? Examining the Role of Third-Person Effects and Free Speech Support in Shaping Moderation Preferences](https://mlsanity.com/paper/2301.02208/) (2023)
- [Identity-related Speech Suppression in Generative AI Content Moderation](https://mlsanity.com/paper/2409.13725/) (2024)
- [SS-BERT: Mitigating Identity Terms Bias in Toxic Comment Classification by Utilising the Notion of "Subjectivity" and "Identity Terms"](https://mlsanity.com/paper/2109.02691/) (2021)
- [Personalized Content Moderation and Emergent Outcomes](https://mlsanity.com/paper/2405.09640/) (2024)
- [Towards Low-Resource Alignment to Diverse Perspectives with Sparse Feedback](https://mlsanity.com/paper/2510.16257/) (2025)

## Abstract (arXiv)

Reclaimed slur usage is a common and meaningful practice online for many marginalized communities. It serves as a source of solidarity, identity, and shared experience. However, contemporary automated and AI-based moderation tools for online content largely fail to distinguish between reclaimed and hateful uses of slurs, resulting in the suppression of marginalized voices. In this work, we use quantitative and qualitative methods to examine the attitudes of social media users in LGBTQIA+, Black, and women communities around reclaimed slurs targeting our focus groups including the f-word, n-word, and b-word. With social media users from these communities, we collect and analyze an annotated online slur usage corpus. The corpus includes annotators' perceptions of whether an online text containing a slur should be flagged as hate speech, as well as contextual features of the slur usage. Across all communities and annotation questions, we observe low inter-annotator agreement, indicating substantial disagreement among in-group annotators. This is compounded by the fact that, absent clear contextual signals of identity and intent, even in-group members may disagree on how to interpret reclaimed slur usage online. Semi-structured interviews with annotators suggest that differences in lived experience and personal history contribute to this variation as well. We find poor alignment between annotator judgments and automated hate speech assessments produced by Perspective API. We further observe that certain features of a text such as whether the slur usage was derogatory and if the slur was targeted at oneself are more associated with whether annotators report the text as hate speech. Together, these findings highlight the inherent subjectivity and contextual nature of how marginalized communities interpret slurs online.