# Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media

arXiv: https://arxiv.org/abs/2104.06999 · HTML: https://mlsanity.com/paper/2104.06999/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Reducing Gender Bias in Abusive Language Detection](https://mlsanity.com/paper/1808.07231/) (2018)
- [Counterfactual Fairness in Text Classification through Robustness](https://mlsanity.com/paper/1809.10610/) (2018)
- [Re-imagining Algorithmic Fairness in India and Beyond](https://mlsanity.com/paper/2101.09995/) (2021)
- [Deceiving Google's Perspective API Built for Detecting Toxic Comments](https://mlsanity.com/paper/1702.08138/) (2017)
- [Challenges in Automated Debiasing for Toxic Language Detection](https://mlsanity.com/paper/2102.00086/) (2021)
- [Perturbation Sensitivity Analysis to Detect Unintended Model Biases](https://mlsanity.com/paper/1910.04210/) (2019)
- [All You Need is "Love": Evading Hate-speech Detection](https://mlsanity.com/paper/1808.09115/) (2018)
- [Challenges for Toxic Comment Classification: An In-Depth Error Analysis](https://mlsanity.com/paper/1809.07572/) (2018)
- ["You are grounded!": Latent Name Artifacts in Pre-trained Language Models](https://mlsanity.com/paper/2004.03012/) (2020)
- [Towards Robust Toxic Content Classification](https://mlsanity.com/paper/1912.06872/) (2019)
- [Fair Hate Speech Detection through Evaluation of Social Group Counterfactuals](https://mlsanity.com/paper/2010.12779/) (2020)
- [Automatically Inferring Gender Associations from Language](https://mlsanity.com/paper/1909.00091/) (2019)
- [Noisy Text Data: Achilles' Heel of BERT](https://mlsanity.com/paper/2003.12932/) (2020)

## Built on by (most cited first, 30 of 31)

- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [NLPositionality: Characterizing Design Biases of Datasets and Models](https://mlsanity.com/paper/2306.01943/) (2023)
- [Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark](https://mlsanity.com/paper/2305.14938/) (2023)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation](https://mlsanity.com/paper/2206.08931/) (2022)
- [Re-contextualizing Fairness in NLP: The Case of India](https://mlsanity.com/paper/2209.12226/) (2022)
- [Socially Aware Bias Measurements for Hindi Language Representations](https://mlsanity.com/paper/2110.07871/) (2021)
- [Geographic and Geopolitical Biases of Language Models](https://mlsanity.com/paper/2212.10408/) (2022)
- [From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets](https://mlsanity.com/paper/2404.17874/) (2024)
- [How Far Can We Extract Diverse Perspectives from Large Language Models?](https://mlsanity.com/paper/2311.09799/) (2023)
- [Cultural Incongruencies in Artificial Intelligence](https://mlsanity.com/paper/2211.13069/) (2022)
- [Cross-Cultural Transfer Learning for Chinese Offensive Language Detection](https://mlsanity.com/paper/2303.17927/) (2023)
- [NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps](https://mlsanity.com/paper/2404.01651/) (2024)
- [Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features](https://mlsanity.com/paper/2310.06458/) (2023)
- [Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive](https://mlsanity.com/paper/2301.12534/) (2023)
- [HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on Twitter](https://mlsanity.com/paper/2411.15462/) (2024)
- [Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations](https://mlsanity.com/paper/2503.01623/) (2025)
- [Discovering Differences in the Representation of People using Contextualized Semantic Axes](https://mlsanity.com/paper/2210.12170/) (2022)
- [Making Intelligence: Ethical Values in IQ and ML Benchmarks](https://mlsanity.com/paper/2209.00692/) (2022)
- [NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative Data](https://mlsanity.com/paper/2403.19260/) (2024)
- [Pre-Trained Language Models Represent Some Geographic Populations Better Than Others](https://mlsanity.com/paper/2403.11025/) (2024)
- [I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models](https://mlsanity.com/paper/2402.10436/) (2024)
- [Regional Bias in Large Language Models](https://mlsanity.com/paper/2601.16349/) (2026)
- [Are Akpans Trick or Treat: Unveiling Helpful Biases in Assistant Systems](https://mlsanity.com/paper/2205.12554/) (2022)
- [A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter](https://mlsanity.com/paper/2210.06351/) (2022)
- [Towards Generalized Offensive Language Identification](https://mlsanity.com/paper/2407.18738/) (2024)
- [Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructure](https://mlsanity.com/paper/2604.25580/) (2026)
- [Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEM](https://mlsanity.com/paper/2604.08425/) (2026)
- [Mental Health Disorder Detection Beyond Social Media: A Systematic Review of Available Datasets](https://mlsanity.com/paper/2607.03540/) (2026)
- [When Noise Fabricates Bias: The Fragility of LLM-as-a-Judge Bias Measurement under Noisy Text](https://mlsanity.com/paper/2609.11067/) (2026)

## Abstract (arXiv)

Online social media platforms increasingly rely on Natural Language Processing (NLP) techniques to detect abusive content at scale in order to mitigate the harms it causes to their users. However, these techniques suffer from various sampling and association biases present in training data, often resulting in sub-par performance on content relevant to marginalized groups, potentially furthering disproportionate harms towards them. Studies on such biases so far have focused on only a handful of axes of disparities and subgroups that have annotations/lexicons available. Consequently, biases concerning non-Western contexts are largely ignored in the literature. In this paper, we introduce a weakly supervised method to robustly detect lexical biases in broader geocultural contexts. Through a case study on a publicly available toxicity detection model, we demonstrate that our method identifies salient groups of cross-geographic errors, and, in a follow up, demonstrate that these groupings reflect human judgments of offensive and inoffensive language in those geographic contexts. We also conduct analysis of a model trained on a dataset with ground truth labels to better understand these biases, and present preliminary mitigation experiments.