# From 'Showgirls' to 'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMs

arXiv: https://arxiv.org/abs/2407.04434 · HTML: https://mlsanity.com/paper/2407.04434/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 5 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- ["I'm fully who I am": Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation](https://mlsanity.com/paper/2305.09941/) (2023)
- [NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternatives](https://mlsanity.com/paper/2109.06105/) (2021)
- [They, Them, Theirs: Rewriting with Gender-Neutral English](https://mlsanity.com/paper/2102.06788/) (2021)
- [The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendations](https://mlsanity.com/paper/2308.02053/) (2023)
- [Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting](https://mlsanity.com/paper/2110.05367/) (2021)
- [On Evaluating and Mitigating Gender Biases in Multilingual Settings](https://mlsanity.com/paper/2307.01503/) (2023)
- [Choose Your Lenses: Flaws in Gender Bias Evaluation](https://mlsanity.com/paper/2210.11471/) (2022)
- [Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions](https://mlsanity.com/paper/2306.04597/) (2023)
- [Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challenges](https://mlsanity.com/paper/2301.10075/) (2023)
- [How Reliable are Model Diagnostics?](https://mlsanity.com/paper/2105.05641/) (2021)
- [Gender-Inclusive Grammatical Error Correction through Augmentation](https://mlsanity.com/paper/2306.07415/) (2023)

## Built on by (most cited first, 15 of 15)

- ['Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMs](https://mlsanity.com/paper/2409.13484/) (2024)
- [Do language models practice what they preach? Examining language ideologies about gendered language reform encoded in LLMs](https://mlsanity.com/paper/2409.13852/) (2024)
- [Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation](https://mlsanity.com/paper/2412.14050/) (2024)
- [On the Mutual Influence of Gender and Occupation in LLM Representations](https://mlsanity.com/paper/2503.06792/) (2025)
- [An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation](https://mlsanity.com/paper/2504.11934/) (2025)
- [Adapting Psycholinguistic Research for LLMs: Gender-inclusive Language in a Coreference Context](https://mlsanity.com/paper/2502.13120/) (2025)
- [The LLM Wears Prada: Analysing Gender Bias and Stereotypes through Online Shopping Data](https://mlsanity.com/paper/2504.01951/) (2025)
- [Analyzing values about gendered language reform in LLMs' revisions](https://mlsanity.com/paper/2505.21378/) (2025)
- [GeNRe: A French Gender-Neutral Rewriting System Using Collective Nouns](https://mlsanity.com/paper/2505.23630/) (2025)
- [Auto-Search and Refinement: An Automated Framework for Gender Bias Mitigation in Large Language Models](https://mlsanity.com/paper/2502.11559/) (2025)
- [FAIRTOPIA: Envisioning Multi-Agent Guardianship for Disrupting Unfair AI Pipelines](https://mlsanity.com/paper/2506.09107/) (2025)
- [Fair Representation in Parliamentary Summaries: Measuring and Mitigating Inclusion Bias](https://mlsanity.com/paper/2507.14221/) (2025)
- [Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs](https://mlsanity.com/paper/2509.13480/) (2025)
- [Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation](https://mlsanity.com/paper/2512.10734/) (2025)
- [ViD: Vision-Dominant Gender Bias Mitigation for Large Vision-Language Models](https://mlsanity.com/paper/2609.16647/) (2026)

## Abstract (arXiv)

Gender bias is not only prevalent in Large Language Models (LLMs) and their training data, but also firmly ingrained into the structural aspects of language itself. Therefore, adapting linguistic structures within LLM training data to promote gender-inclusivity can make gender representations within the model more inclusive. The focus of our work are gender-exclusive affixes in English, such as in 'show-girl' or 'man-cave', which can perpetuate gender stereotypes and binary conceptions of gender. We use an LLM training dataset to compile a catalogue of 692 gender-exclusive terms along with gender-neutral variants and from this, develop a gender-inclusive fine-tuning dataset, the 'Tiny Heap'. Fine-tuning three different LLMs with this dataset, we observe an overall reduction in gender-stereotyping tendencies across the models. Our approach provides a practical method for enhancing gender inclusivity in LLM training data and contributes to incorporating queer-feminist linguistic activism in bias mitigation research in NLP.