mlsanityData to 7 Oct 2026

Gender Bias in Meta-Embeddings

arXiv 2205.09867 ↗First posted 19 May 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Different methods have been proposed to develop meta-embeddings from a given set of source embeddings. However, the source embeddings can contain unfair gender-related biases, and how these influence the meta-embeddings has not been studied yet. We study the gender bias in meta-embeddings created under three different settings: (1) meta-embedding multiple sources without performing any debiasing (Multi-Source No-Debiasing), (2) meta-embedding multiple sources debiased by a single method (Multi-Source Single-Debiasing), and (3) meta-embedding a single source debiased by different methods (Single-Source Multi-Debiasing). Our experimental results show that meta-embedding amplifies the gender biases compared to input source embeddings. We find that debiasing not only the sources but also their meta-embedding is needed to mitigate those biases. Moreover, we propose a novel debiasing method based on meta-embedding learning where we use multiple debiasing methods on a single source embedding and then create a single unbiased meta-embedding.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 38 of its 63 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

38 of 63 references.

References · 38

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  3. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  4. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  5. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  7. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  8. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  9. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  10. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  11. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  12. 2019Gender Bias in Contextualized Word Embeddingscited by 203
26 more
  1. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 170
  2. 2018Learning Gender-Neutral Word Embeddingscited by 169
  3. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 166
  4. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  5. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  6. 2017Controllable Invariance through Adversarial Feature Learningcited by 127
  7. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  8. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  9. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  10. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  11. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  12. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  13. 2019Gender-preserving Debiasing for Pre-trained Word Embeddingscited by 61
  14. 2021Unmasking the Mask -- Evaluating Social Biases in Masked Language Modelscited by 60
  15. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  16. 2020Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigationcited by 36
  17. 2021Dictionary-based Debiasing of Pre-trained Word Embeddingscited by 24
  18. 2022Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddingscited by 18
  19. 2018Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddingscited by 15
  20. 2020Autoencoding Improves Pre-trained Word Embeddingscited by 11
  21. 2017Think Globally, Embed Locally --- Locally Linear Meta-embedding of Wordscited by 8
  22. 2019Sentence Meta-Embeddings for Unsupervised Semantic Textual Similaritycited by 8
  23. 2022A Survey on Word Meta-Embedding Learningcited by 7
  24. 2017Linear Ensembles of Word Embedding Modelscited by 6
  25. 2022Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddingscited by 5
  26. 2015Learning Meta-Embeddings by Using Ensembles of Embedding Setscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 0; 2026: 1.

6 papers.

  1. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  2. 2023Uncovering and Quantifying Social Biases in Code Generationcited by 25
  3. 2023Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examplescited by 8
  4. 2024The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Modelscited by 6
  5. 2023Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddingscited by 1
  6. 2026IndicFairFace: Balanced Indian Face Dataset for Auditing and Mitigating Geographical Bias in Vision-Language Modelscited by 0