mlsanityData to 7 Oct 2026

HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models

arXiv 2211.02882 ↗First posted 5 Nov 2022cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Fairness has become a trending topic in natural language processing (NLP), which addresses biases targeting certain social groups such as genders and religions. However, regional bias in language models (LMs), a long-standing global discrimination problem, still remains unexplored. This paper bridges the gap by analysing the regional bias learned by the pre-trained language models that are broadly used in NLP tasks. In addition to verifying the existence of regional bias in LMs, we find that the biases on regional groups can be strongly influenced by the geographical clustering of the groups. We accordingly propose a HiErarchical Regional Bias evaluation method (HERB) utilising the information from the sub-region clusters to quantify the bias in pre-trained LMs. Experiments show that our hierarchical metric can effectively evaluate the regional bias with respect to comprehensive topics and measure the potential regional bias that can be propagated to downstream tasks. Our codes are available at https://github.com/Bernard-Yang/HERB.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 5 Nov 2022.

Builds on

28 of 45 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  6. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  7. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  8. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  9. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  10. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  11. 2019Measuring Bias in Contextualized Word Representationscited by 275
  12. 2018A Simple Method for Commonsense Reasoningcited by 223
16 more
  1. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  2. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  3. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  4. 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
  5. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  6. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  7. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
  8. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  9. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  10. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  11. 2020Demoting Racial Bias in Hate Speech Detectioncited by 62
  12. 2021Unmasking the Mask -- Evaluating Social Biases in Masked Language Modelscited by 60
  13. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  14. 2019Examining Gender Bias in Languages with Grammatical Gendercited by 40
  15. 2021Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Modelscited by 20
  16. 2021Gender Bias Hidden Behind Chinese Word Embeddings: The Case of Chinese Adjectivescited by 7

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 1; 2025: 1.

8 papers.

  1. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  2. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  3. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  4. 2023Uncovering and Quantifying Social Biases in Code Generationcited by 25
  5. 2024The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Modelscited by 6
  6. 2023Uncovering and Categorizing Social Biases in Text-to-SQLcited by 5
  7. 2023Detecting Spoilers in Movie Reviews with External Movie Knowledge and User Networkscited by 4
  8. 2025Overview of the NLPCC 2025 Shared Task: Gender Bias Mitigation Challengecited by 0