# GKnow: Measuring the Entanglement of Gender Bias and Factual Gender

arXiv: https://arxiv.org/abs/2605.12299 · HTML: https://mlsanity.com/paper/2605.12299/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Linearity of Relation Decoding in Transformer Language Models](https://mlsanity.com/paper/2308.09124/) (2023)
- [Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias](https://mlsanity.com/paper/2004.12265/) (2020)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [Information Flow Routes: Automatically Interpreting Language Models at Scale](https://mlsanity.com/paper/2403.00824/) (2024)
- [Gender-preserving Debiasing for Pre-trained Word Embeddings](https://mlsanity.com/paper/1906.00742/) (2019)
- [Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons](https://mlsanity.com/paper/2308.13198/) (2023)
- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [What does the Knowledge Neuron Thesis Have to do with Knowledge?](https://mlsanity.com/paper/2405.02421/) (2024)
- [Choose Your Lenses: Flaws in Gender Bias Evaluation](https://mlsanity.com/paper/2210.11471/) (2022)
- [Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias](https://mlsanity.com/paper/2212.10678/) (2022)
- [Debiasing Algorithm through Model Adaptation](https://mlsanity.com/paper/2310.18913/) (2023)
- [Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model](https://mlsanity.com/paper/2310.12611/) (2023)
- [Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information](https://mlsanity.com/paper/2206.10744/) (2022)
- [Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective](https://mlsanity.com/paper/2506.05166/) (2025)
- [Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing](https://mlsanity.com/paper/2501.14457/) (2025)
- [Are formal and functional linguistic mechanisms dissociated in language models?](https://mlsanity.com/paper/2503.11302/) (2025)
- [DiFair: A Benchmark for Disentangled Assessment of Gender Knowledge and Bias](https://mlsanity.com/paper/2310.14329/) (2023)
- [Unveiling Language Competence Neurons: A Psycholinguistic Approach to Model Interpretability](https://mlsanity.com/paper/2409.15827/) (2024)

## Abstract (arXiv)

Recent works have analyzed the impact of individual components of neural networks on gendered predictions, often with a focus on mitigating gender bias. However, mechanistic interpretations of gender tend to (i) focus on a very specific gender-related task, such as gendered pronoun prediction, or (ii) fail to distinguish between the production of factually gendered outputs (the correct assumption of gender given a word that carries gender as a semantic property) and gender biased outputs (based on a stereotype). To address these issues, we curate gknow, a benchmark to assess gender knowledge and gender bias in language models across different types of gender-related predictions. gknow allows us to identify and analyze circuits and individual neurons responsible for gendered predictions. We test the impact of neuron ablation on benchmarks for disentangling stereotypical and factual gender (DiFair and the test set of GKnow), as well as StereoSet. Results show that gender bias and factual gender are severely entangled on the level of both circuits and neurons, entailing that ablation is an unreliable debiasing method. Furthermore, we show that benchmarks for evaluating gender bias can hide the decrease in factual gender knowledge that accompanies neuron ablation. We curate GKnow as a contribution to the continuous development of robust gender bias benchmarks.