mlsanityData to 7 Oct 2026

How Prevalent is Gender Bias in ChatGPT? -- Exploring German and English ChatGPT Responses

arXiv 2310.03031 ↗First posted 21 Sep 2023cs.CL cs.AI cs.CYcited by 5

Not on the map. Why?

Abstract (arXiv)

With the introduction of ChatGPT, OpenAI made large language models (LLM) accessible to users with limited IT expertise. However, users with no background in natural language processing (NLP) might lack a proper understanding of LLMs. Thus the awareness of their inherent limitations, and therefore will take the systems' output at face value. In this paper, we systematically analyse prompts and the generated responses to identify possible problematic issues with a special focus on gender biases, which users need to be aware of when processing the system's output. We explore how ChatGPT reacts in English and German if prompted to answer from a female, male, or neutral perspective. In an in-depth investigation, we examine selected prompts and analyse to what extent responses differ if the system is prompted several times in an identical way. On this basis, we show that ChatGPT is indeed useful for helping non-IT users draft texts for their daily work. However, it is absolutely crucial to thoroughly check the system's responses for biases as well as for syntactic and grammatical mistakes.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 10 of its 25 references that are in the map. First posted on arXiv 21 Sep 2023.

Builds on

10 of 25 references.

References · 10

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Unsupervised Discovery of Implicit Gender Biascited by 34
  6. 2015It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopediacited by 30
  7. 2016Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Communitycited by 21
  8. 2015First Women, Second Sex: Gender Bias in Wikipediacited by 11
  9. 2016Tie-breaker: Using language models to quantify gender bias in sports journalismcited by 11
  10. 2016Gendered Conversation in a Social Game-Streaming Platformcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  2. 2023Survey on AI Ethics: A Socio-technical Perspectivecited by 4
  3. 2024How far can bias go? Tracing bias from pretraining data to alignmentcited by 4
  4. 2024Towards detecting unanticipated bias in Large Language Modelscited by 3
  5. 2025Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Languagecited by 3