# Disentangling Perceptions of Offensiveness: Cultural and Moral Correlates

arXiv: https://arxiv.org/abs/2312.06861 · HTML: https://mlsanity.com/paper/2312.06861/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 101 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 30 of 36)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models](https://mlsanity.com/paper/2111.02840/) (2021)
- [Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment](https://mlsanity.com/paper/2308.09662/) (2023)
- [Challenges and Strategies in Cross-Cultural NLP](https://mlsanity.com/paper/2203.10020/) (2022)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure](https://mlsanity.com/paper/2010.13561/) (2020)
- [Toward a Perspectivist Turn in Ground Truthing for Predictive Computing](https://mlsanity.com/paper/2109.04270/) (2021)
- [On Releasing Annotator-Level Labels and Information in Datasets](https://mlsanity.com/paper/2110.05699/) (2021)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [Explore, Establish, Exploit: Red Teaming Language Models from Scratch](https://mlsanity.com/paper/2306.09442/) (2023)
- [NLPositionality: Characterizing Design Biases of Datasets and Models](https://mlsanity.com/paper/2306.01943/) (2023)
- [When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks](https://mlsanity.com/paper/2305.06626/) (2023)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [AI model GPT-3 (dis)informs us better than humans](https://mlsanity.com/paper/2301.11924/) (2023)
- [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation](https://mlsanity.com/paper/2205.00501/) (2022)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation](https://mlsanity.com/paper/2206.08931/) (2022)
- [AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia](https://mlsanity.com/paper/2305.11844/) (2023)
- [Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information](https://mlsanity.com/paper/2301.05036/) (2023)
- [How Different Groups Prioritize Ethical Values for Responsible AI](https://mlsanity.com/paper/2205.07722/) (2022)

## Built on by (most cited first, 26 of 26)

- [AI Alignment at Your Discretion](https://mlsanity.com/paper/2502.10441/) (2025)
- [Beyond Consensus: Perspectivist Modeling and Evaluation of Annotator Disagreement in NLP](https://mlsanity.com/paper/2601.09065/) (2026)
- [Evaluating Moral Beliefs across LLMs through a Pluralistic Framework](https://mlsanity.com/paper/2411.03665/) (2024)
- [Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models](https://mlsanity.com/paper/2507.13383/) (2025)
- [Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations](https://mlsanity.com/paper/2403.06009/) (2024)
- [Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives](https://mlsanity.com/paper/2508.02853/) (2025)
- [Re-examining Sexism and Misogyny Classification with Annotator Attitudes](https://mlsanity.com/paper/2410.03543/) (2024)
- [Value Compass Benchmarks: A Platform for Fundamental and Validated Evaluation of LLMs Values](https://mlsanity.com/paper/2501.07071/) (2025)
- [Un-Straightening Generative AI: How Queer Artists Surface and Challenge the Normativity of Generative AI Models](https://mlsanity.com/paper/2503.09805/) (2025)
- [JailNewsBench: Multi-Lingual and Regional Benchmark for Fake News Generation under Jailbreak Attacks](https://mlsanity.com/paper/2603.01291/) (2026)
- [Self-Explaining Hate Speech Detection with Moral Rationales](https://mlsanity.com/paper/2601.03481/) (2026)
- [Adaptable Moral Stances of Large Language Models on Sexist Content: Implications for Society and Gender Discourse](https://mlsanity.com/paper/2410.00175/) (2024)
- [Insights on Disagreement Patterns in Multimodal Safety Perception across Diverse Rater Groups](https://mlsanity.com/paper/2410.17032/) (2024)
- [Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures](https://mlsanity.com/paper/2502.17710/) (2025)
- [Trustworthy and Responsible AI for Human-Centric Autonomous Decision-Making Systems](https://mlsanity.com/paper/2408.15550/) (2024)
- [Reflect: Transparent Principle-Guided Reasoning for Constitutional Alignment at Scale](https://mlsanity.com/paper/2601.18730/) (2026)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models](https://mlsanity.com/paper/2410.19812/) (2024)
- [That is Unacceptable: the Moral Foundations of Canceling](https://mlsanity.com/paper/2503.05720/) (2025)
- [Language, Culture, and Ideology: Personalizing Offensiveness Detection in Political Tweets with Reasoning LLMs](https://mlsanity.com/paper/2510.02351/) (2025)
- [Labels have Human Values: Value Calibration of Subjective Tasks](https://mlsanity.com/paper/2601.06631/) (2026)
- [Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring](https://mlsanity.com/paper/2501.07324/) (2025)
- [Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEM](https://mlsanity.com/paper/2604.08425/) (2026)
- [The Ghost Annotator: a Framework to Explore Human Label Variation in Content Moderation through Conformal Prediction](https://mlsanity.com/paper/2606.02911/) (2026)
- [Epistemic Injustice in Language Models: An Audit of Pretraining Filters and Guardrails](https://mlsanity.com/paper/2606.05936/) (2026)
- [How Ethos and Pathos Appeals Resonate in Reader Interpretations of Social Media Messages](https://mlsanity.com/paper/2607.00873/) (2026)

## Abstract (arXiv)

Perception of offensiveness is inherently subjective, shaped by the lived experiences and socio-cultural values of the perceivers. Recent years have seen substantial efforts to build AI-based tools that can detect offensive language at scale, as a means to moderate social media platforms, and to ensure safety of conversational AI technologies such as ChatGPT and Bard. However, existing approaches treat this task as a technical endeavor, built on top of data annotated for offensiveness by a global crowd workforce without any attention to the crowd workers' provenance or the values their perceptions reflect. We argue that cultural and psychological factors play a vital role in the cognitive processing of offensiveness, which is critical to consider in this context. We re-frame the task of determining offensiveness as essentially a matter of moral judgment -- deciding the boundaries of ethically wrong vs. right language within an implied set of socio-cultural norms. Through a large-scale cross-cultural study based on 4309 participants from 21 countries across 8 cultural regions, we demonstrate substantial cross-cultural differences in perceptions of offensiveness. More importantly, we find that individual moral values play a crucial role in shaping these variations: moral concerns about Care and Purity are significant mediating factors driving cross-cultural differences. These insights are of crucial importance as we build AI models for the pluralistic world, where the values they espouse should aim to respect and account for moral values in diverse geo-cultural contexts.