mlsanityData to 7 Oct 2026

Estimating Grammatical Gender Directions in Contextual Embeddings under Controlled and Natural Contexts

arXiv 2606.30152 ↗First posted 29 Jun 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Contextual language models conflate grammatical gender and social semantic bias in gendered languages such as Spanish. Existing gender debiasing approaches only operate on static word embeddings leaving contextual representations unexplored for this two dimensional gender disentanglement. To address the this issue, we make the first attempt to disentangle grammatical gender from semantic contamination for contextual embeddings. We construct both controlled templates and natural Wikipedia contexts to build balanced datasets of inanimate nouns, and design a framework equipped with centroid, Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) gender direction estimators as well as contamination-aware weighting strategies. A set of dual-objective evaluation metrics is proposed to balance the suppression of grammatical gender leakage on inanimate nouns and the preservation of semantic gender distinctions for occupation terms. The results reveal that unweighted controlled contexts yield the purest grammatical gender direction, and the centroid estimator achieves better performance than discriminative baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

31 of 45 references.

References · 31

  1. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  3. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  5. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  6. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  7. 2019CamemBERT: a Tasty French Language Modelcited by 368
  8. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  9. 2019Measuring Bias in Contextualized Word Representationscited by 275
  10. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  11. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  12. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
19 more
  1. 2023A Survey on Fairness in Large Language Modelscited by 91
  2. 2023Spanish Pre-trained BERT Model and Evaluation Datacited by 81
  3. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  4. 2019Examining Gender Bias in Languages with Grammatical Gendercited by 40
  5. 2020OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddingscited by 40
  6. 2024Discovering Low-rank Subspaces for Language-agnostic Multilingual Representationscited by 18
  7. 2019How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?cited by 14
  8. 2020Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddingscited by 9
  9. 2025Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Modelscited by 7
  10. 2022Measuring Gender Bias in Word Embeddings of Gendered Languages Requires Disentangling Grammatical Gender Signalscited by 5
  11. 2024Leveraging Large Language Models to Measure Gender Representation Bias in Gendered Language Corporacited by 5
  12. 2024What an Elegant Bridge: Multilingual LLMs are Biased Similarly in Different Languagescited by 5
  13. 2025EuroGEST: Investigating gender stereotypes in multilingual language modelscited by 4
  14. 2025Beyond Content: How Grammatical Gender Shapes Visual Representation in Text-to-Image Modelscited by 4
  15. 2023The Causal Influence of Grammatical Gender on Distributional Semanticscited by 3
  16. 2024What is "Typological Diversity" in NLP?cited by 3
  17. 2024Investigating grammatical abstraction in language models using few-shot learning of novel noun gendercited by 2
  18. 2024Mitigating Gender Bias in Contextual Word Embeddingscited by 1
  19. 2026RobustDebias: Debiasing Language Models using Distributionally Robust Optimizationcited by 1

Built on by

Not cited yet.