# Bias in Decision-Making for AI's Ethical Dilemmas: A Comparative Study of ChatGPT and Claude

arXiv: https://arxiv.org/abs/2501.10484 · HTML: https://mlsanity.com/paper/2501.10484/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 44 references that are in the map.
- First posted on arXiv 17 Jan 2025.

## Builds on (most cited first, 7 of 7)

- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [The Measure and Mismeasure of Fairness](https://mlsanity.com/paper/1808.00023/) (2018)
- [The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendations](https://mlsanity.com/paper/2308.02053/) (2023)
- [Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets](https://mlsanity.com/paper/2410.07991/) (2024)
- [The Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairness](https://mlsanity.com/paper/2307.05842/) (2023)
- [Are LLMs Socially Adaptive? Contrasting Belief Evolution in Large Language Models and Humans](https://mlsanity.com/paper/2410.10398/) (2024)
- [Gender Bias in LLM-generated Interview Responses](https://mlsanity.com/paper/2410.20739/) (2024)

## Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have enabled human-like responses across various tasks, raising questions about their ethical decision-making capabilities and potential biases. This study systematically evaluates how nine popular LLMs (both open-source and closed-source) respond to ethical dilemmas involving protected attributes. Across 50,400 trials spanning single and intersectional attribute combinations in four dilemma scenarios (protective vs. harmful), we assess models' ethical preferences, sensitivity, stability, and clustering patterns. Results reveal significant biases in protected attributes in all models, with differing preferences depending on model type and dilemma context. Notably, open-source LLMs show stronger preferences for marginalized groups and greater sensitivity in harmful scenarios, while closed-source models are more selective in protective situations and tend to favor mainstream groups. We also find that ethical behavior varies across dilemma types: LLMs maintain consistent patterns in protective scenarios but respond with more diverse and cognitively demanding decisions in harmful ones. Furthermore, models display more pronounced ethical tendencies under intersectional conditions than in single-attribute settings, suggesting that complex inputs reveal deeper biases. These findings highlight the need for multi-dimensional, context-aware evaluation of LLMs' ethical behavior and offer a systematic evaluation and approach to understanding and addressing fairness in LLM decision-making.