# Measuring Gender Bias in West Slavic Language Models

arXiv: https://arxiv.org/abs/2304.05783 · HTML: https://mlsanity.com/paper/2304.05783/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias](https://mlsanity.com/paper/2010.14534/) (2020)
- [On Measuring Gender Bias in Translation of Gender-neutral Pronouns](https://mlsanity.com/paper/1905.11684/) (2019)
- [They, Them, Theirs: Rewriting with Gender-Neutral English](https://mlsanity.com/paper/2102.06788/) (2021)
- [Czert -- Czech BERT-like Model for Language Representation](https://mlsanity.com/paper/2103.13031/) (2021)
- [Pre-training Polish Transformer-based Language Models at Scale](https://mlsanity.com/paper/2006.04229/) (2020)
- [Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Models](https://mlsanity.com/paper/2104.07505/) (2021)
- [Entity-Centric Contextual Affective Analysis](https://mlsanity.com/paper/1906.01762/) (2019)
- [SlovakBERT: Slovak Masked Language Model](https://mlsanity.com/paper/2109.15254/) (2021)
- [Measuring Harmful Representations in Scandinavian Language Models](https://mlsanity.com/paper/2211.11678/) (2022)
- [Quantifying Gender Biases Towards Politicians on Reddit](https://mlsanity.com/paper/2112.12014/) (2021)

## Built on by (most cited first, 10 of 10)

- [Social Bias Probing: Fairness Benchmarking for Language Models](https://mlsanity.com/paper/2311.09090/) (2023)
- [Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications](https://mlsanity.com/paper/2307.09162/) (2023)
- [What's Not Said Still Hurts: A Description-Based Evaluation Framework for Measuring Social Bias in LLMs](https://mlsanity.com/paper/2502.19749/) (2025)
- [Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms](https://mlsanity.com/paper/2503.04372/) (2025)
- [How Gender Interacts with Political Values: A Case Study on Czech BERT Models](https://mlsanity.com/paper/2403.13514/) (2024)
- [Transforming Dutch: Debiasing Dutch Coreference Resolution Systems for Non-binary Pronouns](https://mlsanity.com/paper/2405.00134/) (2024)
- [Social Bias in Multilingual Language Models: A Survey](https://mlsanity.com/paper/2508.20201/) (2025)
- [Template-Based Probes Are Imperfect Lenses for Counterfactual Bias Evaluation in LLMs](https://mlsanity.com/paper/2404.03471/) (2024)
- [SESGO: Spanish Evaluation of Stereotypical Generative Outputs](https://mlsanity.com/paper/2509.03329/) (2025)
- [Integrating gender inclusivity into large language models via instruction tuning](https://mlsanity.com/paper/2508.18466/) (2025)

## Abstract (arXiv)

Pre-trained language models have been known to perpetuate biases from the underlying datasets to downstream tasks. However, these findings are predominantly based on monolingual language models for English, whereas there are few investigative studies of biases encoded in language models for languages beyond English. In this paper, we fill this gap by analysing gender bias in West Slavic language models. We introduce the first template-based dataset in Czech, Polish, and Slovak for measuring gender bias towards male, female and non-binary subjects. We complete the sentences using both mono- and multilingual language models and assess their suitability for the masked language modelling objective. Next, we measure gender bias encoded in West Slavic language models by quantifying the toxicity and genderness of the generated words. We find that these language models produce hurtful completions that depend on the subject's gender. Perhaps surprisingly, Czech, Slovak, and Polish language models produce more hurtful completions with men as subjects, which, upon inspection, we find is due to completions being related to violence, death, and sickness.