# How Prevalent is Gender Bias in ChatGPT? -- Exploring German and English ChatGPT Responses

arXiv: https://arxiv.org/abs/2310.03031 · HTML: https://mlsanity.com/paper/2310.03031/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Unsupervised Discovery of Implicit Gender Bias](https://mlsanity.com/paper/2004.08361/) (2020)
- [It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia](https://mlsanity.com/paper/1501.06307/) (2015)
- [Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community](https://mlsanity.com/paper/1603.08832/) (2016)
- [First Women, Second Sex: Gender Bias in Wikipedia](https://mlsanity.com/paper/1502.02341/) (2015)
- [Tie-breaker: Using language models to quantify gender bias in sports journalism](https://mlsanity.com/paper/1607.03895/) (2016)
- [Gendered Conversation in a Social Game-Streaming Platform](https://mlsanity.com/paper/1611.06459/) (2016)

## Built on by (most cited first, 5 of 5)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [How far can bias go? Tracing bias from pretraining data to alignment](https://mlsanity.com/paper/2411.19240/) (2024)
- [Survey on AI Ethics: A Socio-technical Perspective](https://mlsanity.com/paper/2311.17228/) (2023)
- [Towards detecting unanticipated bias in Large Language Models](https://mlsanity.com/paper/2404.02650/) (2024)
- [Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language](https://mlsanity.com/paper/2507.16557/) (2025)

## Abstract (arXiv)

With the introduction of ChatGPT, OpenAI made large language models (LLM) accessible to users with limited IT expertise. However, users with no background in natural language processing (NLP) might lack a proper understanding of LLMs. Thus the awareness of their inherent limitations, and therefore will take the systems' output at face value. In this paper, we systematically analyse prompts and the generated responses to identify possible problematic issues with a special focus on gender biases, which users need to be aware of when processing the system's output. We explore how ChatGPT reacts in English and German if prompted to answer from a female, male, or neutral perspective. In an in-depth investigation, we examine selected prompts and analyse to what extent responses differ if the system is prompted several times in an identical way. On this basis, we show that ChatGPT is indeed useful for helping non-IT users draft texts for their daily work. However, it is absolutely crucial to thoroughly check the system's responses for biases as well as for syntactic and grammatical mistakes.