Speciesist Language and Nonhuman Animal Bias in English Masked Language Models
Not on the map. Why?
Abstract (arXiv)
Various existing studies have analyzed what social biases are inherited by NLP models. These biases may directly or indirectly harm people, therefore previous studies have focused only on human attributes. However, until recently no research on social biases in NLP regarding nonhumans existed. In this paper, we analyze biases to nonhuman animals, i.e. speciesist bias, inherent in English Masked Language Models such as BERT. We analyzed speciesist bias against 46 animal names using template-based and corpus-extracted sentences containing speciesist (or non-speciesist) language. We found that pre-trained masked language models tend to associate harmful words with nonhuman animals and have a bias toward using speciesist language for some nonhuman animal names. Our code for reproducing the experiments will be made available on GitHub.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 48 of its 91 references that are in the map. First posted on arXiv 10 Mar 2022.
Builds on
48 of 91 references.
References · 48
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
- 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
- 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
36 more
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
- 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
- 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
- 2019On Measuring Social Biases in Sentence Encoderscited by 397
- 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
- 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
- 2019Measuring Bias in Contextualized Word Representationscited by 275
- 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
- 2018A Simple Method for Commonsense Reasoningcited by 223
- 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
- 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
- 2019Gender Bias in Contextualized Word Embeddingscited by 203
- 2021Societal Biases in Language Generation: Progress and Challengescited by 201
- 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
- 2020Towards Debiasing Sentence Representationscited by 170
- 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
- 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
- 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
- 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
- 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
- 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
- 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
- 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
- 2021Sustainable Modular Debiasing of Language Modelscited by 98
- 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
- 2019Evaluating the Underlying Gender Bias in Contextualized Word Embeddingscited by 94
- 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
- 2021A Survey on Gender Bias in Natural Language Processingcited by 89
- 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
- 2020It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbationscited by 64
- 2019What's in a Name? Reducing Bias in Bios without Access to Protected Attributescited by 51
- 2020OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddingscited by 40
- 2019A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spacescited by 36
- 2020When do Word Embeddings Accurately Reflect Surveys on our Beliefs About People?cited by 11
- 2022Speciesist bias in AI -- How AI applications perpetuate discrimination and unfair outcomes against animalscited by 11
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 1.
2 papers.