mlsanityData to 7 Oct 2026

Analyzing Gender Representation in Multilingual Models

arXiv 2204.09168 ↗First posted 20 Apr 2022cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Multilingual language models were shown to allow for nontrivial transfer across scripts and languages. In this work, we study the structure of the internal representations that enable this transfer. We focus on the representation of gender distinctions as a practical case study, and examine the extent to which the gender concept is encoded in shared subspaces across different languages. Our analysis shows that gender representations consist of several prominent components that are shared across languages, alongside language-specific components. The existence of language-independent and language-specific components provides an explanation for an intriguing empirical observation we make: while gender classification transfers well across languages, interventions for gender removal, trained on a single language, do not transfer easily to others.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 20 Apr 2022.

Builds on

27 of 32 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2019How multilingual is Multilingual BERT?cited by 642
  6. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  7. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  8. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  9. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  10. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  11. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  12. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
15 more
  1. 2020Multilingual Alignment of Contextual Word Representationscited by 113
  2. 2020Finding Universal Grammatical Relations in Multilingual BERTcited by 108
  3. 2022Linear Adversarial Concept Erasurecited by 98
  4. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 82
  5. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  6. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 61
  7. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 58
  8. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  9. 2019Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsingcited by 46
  10. 2019Examining Gender Bias in Languages with Grammatical Gendercited by 40
  11. 2020It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERTcited by 30
  12. 2019How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?cited by 14
  13. 2020Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizicited by 13
  14. 2020On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbscited by 8
  15. 2021Debiasing Multilingual Word Embeddings: A Case Study of Three Indian Languagescited by 8

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 0; 2024: 4; 2025: 3.

10 papers.

  1. 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19
  2. 2022Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projectioncited by 17
  3. 2024Don't Touch My Diacriticscited by 7
  4. 2024Comparing Explanation Faithfulness between Multilingual and Monolingual Fine-tuned Language Modelscited by 6
  5. 2022Undesirable Biases in NLP: Addressing Challenges of Measurementcited by 4
  6. 2024Investigating grammatical abstraction in language models using few-shot learning of novel noun gendercited by 2
  7. 2025Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translationcited by 2
  8. 2024MultiContrievers: Analysis of Dense Retrieval Representationscited by 1
  9. 2025Iterative Multilingual Spectral Attribute Erasurecited by 1
  10. 2025Benchmarking Gender and Political Bias in Large Language Modelscited by 0