# Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors

arXiv: https://arxiv.org/abs/1904.11783 · HTML: https://mlsanity.com/paper/1904.11783/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2019.

## Built on by (most cited first, 30 of 31)

- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [A Survey of Race, Racism, and Anti-Racism in NLP](https://mlsanity.com/paper/2106.11410/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Gender Bias in Masked Language Models for Multiple Languages](https://mlsanity.com/paper/2205.00551/) (2022)
- [Welcome to the Modern World of Pronouns: Identity-Inclusive Natural Language Processing beyond Gender](https://mlsanity.com/paper/2202.11923/) (2022)
- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [Dictionary-based Debiasing of Pre-trained Word Embeddings](https://mlsanity.com/paper/2101.09525/) (2021)
- [Analyzing And Editing Inner Mechanisms Of Backdoored Language Models](https://mlsanity.com/paper/2302.12461/) (2023)
- [Global Voices, Local Biases: Socio-Cultural Prejudices across Languages](https://mlsanity.com/paper/2310.17586/) (2023)
- [Fair and Argumentative Language Modeling for Computational Argumentation](https://mlsanity.com/paper/2204.04026/) (2022)
- [An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models](https://mlsanity.com/paper/2306.04067/) (2023)
- [Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Models](https://mlsanity.com/paper/2108.10379/) (2021)
- [Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases](https://mlsanity.com/paper/2108.06295/) (2021)
- [SocioProbe: What, When, and Where Language Models Learn about Sociodemographics](https://mlsanity.com/paper/2211.04281/) (2022)
- [DebIE: A Platform for Implicit and Explicit Debiasing of Word Embedding Spaces](https://mlsanity.com/paper/2103.06598/) (2021)
- [SWEAT: Scoring Polarization of Topics across Different Corpora](https://mlsanity.com/paper/2109.07231/) (2021)
- [Measuring Gender Bias in Word Embeddings of Gendered Languages Requires Disentangling Grammatical Gender Signals](https://mlsanity.com/paper/2206.01691/) (2022)
- [Bridging Fairness and Environmental Sustainability in Natural Language Processing](https://mlsanity.com/paper/2211.04256/) (2022)
- [How Different Is Stereotypical Bias Across Languages?](https://mlsanity.com/paper/2307.07331/) (2023)
- [A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America](https://mlsanity.com/paper/2207.06591/) (2022)
- [A Bayesian approach to uncertainty in word embedding bias estimation](https://mlsanity.com/paper/2306.09066/) (2023)
- [Racial Bias Trends in the Text of US Legal Opinions](https://mlsanity.com/paper/2307.01693/) (2023)
- [Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionals](https://mlsanity.com/paper/2504.00860/) (2025)
- [Discrimination by LLMs: Cross-lingual Bias Assessment and Mitigation in Decision-Making and Summarisation](https://mlsanity.com/paper/2509.09735/) (2025)

## Abstract (arXiv)

Word embeddings have recently been shown to reflect many of the pronounced societal biases (e.g., gender bias or racial bias). Existing studies are, however, limited in scope and do not investigate the consistency of biases across relevant dimensions like embedding models, types of texts, and different languages. In this work, we present a systematic study of biases encoded in distributional word vector spaces: we analyze how consistent the bias effects are across languages, corpora, and embedding models. Furthermore, we analyze the cross-lingual biases encoded in bilingual embedding spaces, indicative of the effects of bias transfer encompassed in cross-lingual transfer of NLP models. Our study yields some unexpected findings, e.g., that biases can be emphasized or downplayed by different embedding models or that user-generated content may be less biased than encyclopedic text. We hope our work catalyzes bias research in NLP and informs the development of bias reduction techniques.