mlsanityData to 14 Oct 2026

Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive

arXiv 2301.12534 ↗First posted 29 Jan 2023cs.CL cs.CY cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Offensive speech detection is a key component of content moderation. However, what is offensive can be highly subjective. This paper investigates how machine and human moderators disagree on what is offensive when it comes to real-world social web political discourse. We show that (1) there is extensive disagreement among the moderators (humans and machines); and (2) human and large-language-model classifiers are unable to predict how other human raters will respond, based on their political leanings. For (1), we conduct a noise audit at an unprecedented scale that combines both machine and human responses. For (2), we introduce a first-of-its-kind dataset of vicarious offense. Our noise audit reveals that moderation outcomes vary wildly across different machine moderators. Our experiments with human moderators suggest that political leanings combined with sensitive issues affect both first-person and vicarious offense. The dataset is available through https://github.com/Homan-Lab/voiced.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 30 of its 79 references that are in the map. First posted on arXiv 29 Jan 2023.

Builds on

30 of 79 references.

References · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 736
  3. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 407
  4. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  5. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
  6. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 271
  7. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 264
  8. 2019A Benchmark Dataset for Learning to Intervene in Online Hate Speechcited by 129
  9. 2021On Releasing Annotator-Level Labels and Information in Datasetscited by 126
  10. 2017Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisiscited by 96
  11. 2021Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreementcited by 72
  12. 2020Multilingual Offensive Language Identification with Cross-lingual Embeddingscited by 71
18 more
  1. 2019Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootingscited by 67
  2. 2021Designing Toxic Content Classification for a Diversity of Perspectivescited by 65
  3. 2018All You Need is "Love": Evading Hate-speech Detectioncited by 56
  4. 2021Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detectioncited by 45
  5. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  6. 2018CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreementcited by 27
  7. 2020Learning Soft Labels via Meta Learningcited by 27
  8. 2021Pre-train or Annotate? Domain Adaptation with a Constrained Budgetcited by 27
  9. 2018What is Gab? A Bastion of Free Speech or an Alt-Right Echo Chamber?cited by 24
  10. 2020We Don't Speak the Same Language: Interpreting Polarization through Machine Translationcited by 23
  11. 2022Listening to Affected Communities to Define Extreme Speech: Dataset and Experimentscited by 22
  12. 2021Corpus-Level Evaluation for Event QA: The IndiaPoliceEvents Corpus Covering the 2002 Gujarat Violencecited by 19
  13. 2020#Election2020: The First Public Twitter Dataset on the 2020 US Presidential Electioncited by 18
  14. 2021"Short is the Road that Leads from Fear to Hate": Fear Speech in Indian WhatsApp Groupscited by 13
  15. 2020Are Chess Discussions Racist? An Adversarial Hate Speech Data Setcited by 10
  16. 2021Fringe News Networks: Dynamics of US News Viewership following the 2020 Presidential Electioncited by 8
  17. 2020Neighborhood-based Pooling for Population-level Label Distribution Learningcited by 6
  18. 2021"A Virus Has No Religion": Analyzing Islamophobia on Twitter During the COVID-19 Outbreakcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 4; 2026: 4.

16 papers.

Sort:

By area:

  1. 2024Hate Personified: Investigating the role of LLMs in content moderationcited by 25
  2. 2024Rater Cohesion and Quality from a Vicarious Perspectivecited by 8
  3. 2025ProRefine: Inference-Time Prompt Refinement with Textual Feedbackcited by 8
  4. 2024Value Preferences Estimation and Disambiguation in Hybrid Participatory Systemscited by 7
  5. 2024When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summariescited by 7
  6. 2024ARTICLE: Annotator Reliability Through In-Context Learningcited by 6
  7. 2024Infrastructure Ombudsman: Mining Future Failure Concerns from Structural Disaster Responsecited by 4
  8. 2023LLMs and Finetuning: Benchmarking cross-domain performance for hate speech detectioncited by 3
  9. 2025How Inclusively do LMs Perceive Social and Moral Norms?cited by 3
  10. 2025Hope vs. Hate: Understanding User Interactions with LGBTQ+ News Content in Mainstream US News Media through the Lens of Hope Speechcited by 2
  11. 2024Towards Generalized Offensive Language Identificationcited by 1
  12. 2025PromptAug: Fine-grained Conflict Classification Using Data Augmentationcited by 0
4 more
  1. 2026Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEMcited by 0
  2. 2026Investigating Vaccine Buyer's Remorse: Post-Vaccination Decision Regret in COVID-19 Social Media Using Politically Diverse Human Annotationcited by 0
  3. 2026From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotationcited by 0
  4. 2026Thesis Proposal: Toward a Human-Centered and Perspective-Aware Framework for Reproducible ML Evaluation and AI Alignmentcited by 0