# Fairness of ChatGPT

arXiv: https://arxiv.org/abs/2305.18569 · HTML: https://mlsanity.com/paper/2305.18569/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [OpenAGI: When LLM Meets Domain Experts](https://mlsanity.com/paper/2304.04370/) (2023)
- [A survey on datasets for fairness-aware machine learning](https://mlsanity.com/paper/2110.00530/) (2021)
- [Safety Assessment of Chinese Large Language Models](https://mlsanity.com/paper/2304.10436/) (2023)
- [A Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2207.12515/) (2022)
- [UP5: Unbiased Foundation Model for Fairness-aware Recommendation](https://mlsanity.com/paper/2305.12090/) (2023)
- [Fairness in Recommendation: Foundations, Methods and Applications](https://mlsanity.com/paper/2205.13619/) (2022)
- [Modeling Techniques for Machine Learning Fairness: A Survey](https://mlsanity.com/paper/2111.03015/) (2021)
- [Ethical ChatGPT: Concerns, Challenges, and Commandments](https://mlsanity.com/paper/2305.10646/) (2023)
- [Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing](https://mlsanity.com/paper/2304.02017/) (2023)
- [Bias and unfairness in machine learning models: a systematic literature review](https://mlsanity.com/paper/2202.08176/) (2022)
- [Evaluation of group fairness measures in student performance prediction problems](https://mlsanity.com/paper/2208.10625/) (2022)
- [Monotonicity for AI ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and finance](https://mlsanity.com/paper/2301.07060/) (2023)

## Built on by (most cited first, 20 of 20)

- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [VIP5: Towards Multimodal Foundation Models for Recommendation](https://mlsanity.com/paper/2305.14302/) (2023)
- [LLMRec: Benchmarking Large Language Models on Recommendation Task](https://mlsanity.com/paper/2308.12241/) (2023)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [Bias and Fairness in Chatbots: An Overview](https://mlsanity.com/paper/2309.08836/) (2023)
- [CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models](https://mlsanity.com/paper/2407.02408/) (2024)
- [Fairness of ChatGPT and the Role Of Explainable-Guided Prompts](https://mlsanity.com/paper/2307.11761/) (2023)
- [Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like One](https://mlsanity.com/paper/2402.12150/) (2024)
- [No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users](https://mlsanity.com/paper/2410.07589/) (2024)
- [FairCoder: Probing LLM Bias in High-Stakes Decision Making via Coding Tasks](https://mlsanity.com/paper/2501.05396/) (2025)
- [Unveiling Bias in Fairness Evaluations of Large Language Models: A Critical Literature Review of Music and Movie Recommendation Systems](https://mlsanity.com/paper/2401.04057/) (2024)
- [Fairness in Large Language Models in Three Hours](https://mlsanity.com/paper/2408.00992/) (2024)
- [Fair In-Context Learning via Latent Concept Variables](https://mlsanity.com/paper/2411.02671/) (2024)
- [A Review of Fairness and A Practical Guide to Selecting Context-Appropriate Fairness Metrics in Machine Learning](https://mlsanity.com/paper/2411.06624/) (2024)
- [Towards Large Language Models that Benefit for All: Benchmarking Group Fairness in Reward Models](https://mlsanity.com/paper/2503.07806/) (2025)
- [Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review](https://mlsanity.com/paper/2311.14381/) (2023)
- [Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study](https://mlsanity.com/paper/2510.04641/) (2025)
- [Comparing Credit Risk Estimates in the Gen-AI Era](https://mlsanity.com/paper/2506.07754/) (2025)
- [Algorithmic Fairness Over Time: Advances & Prospects](https://mlsanity.com/paper/2610.15015/) (2026)

## Abstract (arXiv)

Understanding and addressing unfairness in LLMs are crucial for responsible AI deployment. However, there is a limited number of quantitative analyses and in-depth studies regarding fairness evaluations in LLMs, especially when applying LLMs to high-stakes fields. This work aims to fill this gap by providing a systematic evaluation of the effectiveness and fairness of LLMs using ChatGPT as a study case. We focus on assessing ChatGPT's performance in high-takes fields including education, criminology, finance and healthcare. To conduct a thorough evaluation, we consider both group fairness and individual fairness metrics. We also observe the disparities in ChatGPT's outputs under a set of biased or unbiased prompts. This work contributes to a deeper understanding of LLMs' fairness performance, facilitates bias mitigation and fosters the development of responsible AI systems.