mlsanityData to 7 Oct 2026

Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings

arXiv 2203.07523 ↗First posted 14 Mar 2022cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word. One sense of an ambiguous word might be socially biased while its other senses remain unbiased. In comparison to the numerous prior work evaluating the social biases in pretrained word embeddings, the biases in sense embeddings have been relatively understudied. We create a benchmark dataset for evaluating the social biases in sense embeddings and propose novel sense-specific bias evaluation measures. We conduct an extensive evaluation of multiple static and contextualised sense embeddings for various types of social biases using the proposed measures. Our experimental results show that even in cases where no biases are found at word-level, there still exist worrying levels of social biases at sense-level, which are often ignored by the word-level bias evaluation measures.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 14 Mar 2022.

Builds on

23 of 36 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  3. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  5. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  6. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  7. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  8. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
  9. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  10. 2019Measuring Bias in Contextualized Word Representationscited by 275
  11. 2018Learning Gender-Neutral Word Embeddingscited by 169
  12. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
11 more
  1. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  2. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  3. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  4. 2019SenseBERT: Driving Some Sense into BERTcited by 73
  5. 2019Gender-preserving Debiasing for Pre-trained Word Embeddingscited by 61
  6. 2021Unmasking the Mask -- Evaluating Social Biases in Masked Language Modelscited by 60
  7. 2019Understanding Undesirable Word Embedding Associationscited by 59
  8. 2019Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguationcited by 40
  9. 2020Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigationcited by 36
  10. 2015Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Spacecited by 35
  11. 2021Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxycited by 6

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 9; 2024: 3; 2025: 2.

18 papers.

Sort:

  1. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  2. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  3. 2023An Efficient Multilingual Language Model Compression through Vocabulary Trimmingcited by 19
  4. 2023Can Word Sense Distribution Detect Semantic Changes of Words?cited by 10
  5. 2023This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Modelscited by 9
  6. 2023Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examplescited by 8
  7. 2023Mitigating Bias for Question Answering Models by Tracking Bias Influencecited by 8
  8. 2023The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimatedcited by 7
  9. 2022Gender Bias in Meta-Embeddingscited by 6
  10. 2023STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Modelscited by 6
  11. 2023Solving NLP Problems through Human-System Collaboration: A Discussion-based Approachcited by 5
  12. 2024The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasingcited by 5
6 more
  1. 2024Eagle: Ethical Dataset Given from Real Interactionscited by 3
  2. 2024Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Modelscited by 3
  3. 2023Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddingscited by 1
  4. 2022On the Curious Case of ℓ2 norm of Sense Embeddingscited by 0
  5. 2025Exploring Gender Bias Beyond Occupational Titlescited by 0
  6. 2025Gamma Mixture Modeling for Cosine Similarity in Small Language Modelscited by 0