mlsanityData to 14 Oct 2026

Evaluating Gender Bias of LLMs in Making Morality Judgements

arXiv 2410.09992 ↗First posted 13 Oct 2024cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown remarkable capabilities in a multitude of Natural Language Processing (NLP) tasks. However, these models are still not immune to limitations such as social biases, especially gender bias. This work investigates whether current closed and open-source LLMs possess gender bias, especially when asked to give moral opinions. To evaluate these models, we curate and introduce a new dataset GenMO (Gender-bias in Morality Opinions) comprising parallel short stories featuring male and female characters respectively. Specifically, we test models from the GPT family (GPT-3.5-turbo, GPT-3.5-turbo-instruct, GPT-4-turbo), Llama 3 and 3.1 families (8B/70B), Mistral-7B and Claude 3 families (Sonnet and Opus). Surprisingly, despite employing safety checks, all production-standard models we tested display significant gender bias with GPT-3.5-turbo giving biased opinions in 24% of the samples. Additionally, all models consistently favour female characters, with GPT showing bias in 68-85% of cases and Llama 3 in around 81-85% instances. Additionally, our study investigates the impact of model parameters on gender bias and explores real-world situations where LLMs reveal biases in moral decision-making.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

26 of 42 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  6. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  7. 2020Aligning AI With Shared Human Valuescited by 723
  8. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  9. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  10. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  11. 2019Measuring Bias in Contextualized Word Representationscited by 276
  12. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 255
14 more
  1. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 253
  2. 2019Evaluating Gender Bias in Machine Translationcited by 249
  3. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  4. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  5. 2021Can Machines Learn Morality? The Delphi Experimentcited by 141
  6. 2019Getting Gender Right in Neural Machine Translationcited by 98
  7. 2019Evaluating the Underlying Gender Bias in Contextualized Word Embeddingscited by 94
  8. 2024What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineeringcited by 90
  9. 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 74
  10. 2024Evaluating Gender Bias in Large Language Models via Chain-of-Thought Promptingcited by 50
  11. 2024Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attributioncited by 40
  12. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  13. 2024Gender Bias in Large Language Models across Multiple Languagescited by 35
  14. 2024Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundationscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025Social Bias Benchmark for Generation: A Comparison of Generation and QA-Based Evaluationscited by 16
  2. 2025The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Modelscited by 7
  3. 2025LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesiscited by 5
  4. 2025Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypescited by 2
  5. 2026Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing Across Diverse Conflict Scenarioscited by 2
  6. 2025GenderBench: Evaluation Suite for Gender Biases in LLMscited by 1
  7. 2025Auto-Search and Refinement: An Automated Framework for Gender Bias Mitigation in Large Language Modelscited by 0
  8. 2025Can we use LLMs to bootstrap reinforcement learning? -- A case study in digital health behavior changecited by 0
  9. 2026Widespread Gender and Pronoun Bias in Moral Judgments Across LLMscited by 0
  10. 2026It Takes One to Bias Them All: Breaking Bad with One-Shot GRPOcited by 0
  11. 2026Wait, am I Being Fair? Characterizing Deductive Stereotyping and Mitigating It with Fair-GCGcited by 0
  12. 2026Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cuescited by 0
3 more
  1. 2026Unequal Verdicts: Investigating Gender Bias in LLM-Based Fake News Detectioncited by 0
  2. 2026One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMscited by 0
  3. 2026Gender bias across LLMs is common and highly heterogeneouscited by 0