# What's in a Name? Auditing Large Language Models for Race and Gender Bias

arXiv: https://arxiv.org/abs/2402.14875 · HTML: https://mlsanity.com/paper/2402.14875/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 11 of 11)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [The Measure and Mismeasure of Fairness](https://mlsanity.com/paper/1808.00023/) (2018)
- [An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction](https://mlsanity.com/paper/2007.10306/) (2020)
- [Algorithmic Fairness in Education](https://mlsanity.com/paper/2007.05443/) (2020)
- [Are Emily and Greg Still More Employable than Lakisha and Jamal? Investigating Algorithmic Hiring Bias in the Era of ChatGPT](https://mlsanity.com/paper/2310.05135/) (2023)
- [Algorithmic Auditing and Social Justice: Lessons from the History of Audit Studies](https://mlsanity.com/paper/2109.06974/) (2021)
- [Affirmative Algorithms: The Legal Grounds for Fairness as Awareness](https://mlsanity.com/paper/2012.14285/) (2020)

## Built on by (most cited first, 30 of 51)

- ["You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations](https://mlsanity.com/paper/2406.12232/) (2024)
- [LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education](https://mlsanity.com/paper/2410.14012/) (2024)
- [Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?](https://mlsanity.com/paper/2406.10486/) (2024)
- [Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge](https://mlsanity.com/paper/2504.07887/) (2025)
- [Presumed Cultural Identity: How Names Shape LLM Responses](https://mlsanity.com/paper/2502.11995/) (2025)
- [Auditing the Use of Language Models to Guide Hiring Decisions](https://mlsanity.com/paper/2404.03086/) (2024)
- [Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)](https://mlsanity.com/paper/2407.12858/) (2024)
- [Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education](https://mlsanity.com/paper/2503.19182/) (2025)
- [Fairness Testing of Large Language Models in Role-Playing](https://mlsanity.com/paper/2411.00585/) (2024)
- [Detecting Prefix Bias in LLM-based Reward Models](https://mlsanity.com/paper/2505.13487/) (2025)
- [Representation Bias in Political Sample Simulations with Large Language Models](https://mlsanity.com/paper/2407.11409/) (2024)
- [DAIQ: Auditing Demographic Attribute Inference from Question in LLMs](https://mlsanity.com/paper/2508.15830/) (2025)
- [Understanding Intrinsic Socioeconomic Biases in Large Language Models](https://mlsanity.com/paper/2405.18662/) (2024)
- [From Superficial Outputs to Superficial Learning: Risks of Large Language Models in Education](https://mlsanity.com/paper/2509.21972/) (2025)
- [All You Need is One: Capsule Prompt Tuning with a Single Vector](https://mlsanity.com/paper/2510.16670/) (2025)
- [More or Less Wrong: A Benchmark for Directional Bias in LLM Comparative Reasoning](https://mlsanity.com/paper/2506.03923/) (2025)
- [Improving LLM Group Fairness on Tabular Data via In-Context Learning](https://mlsanity.com/paper/2412.04642/) (2024)
- [In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations](https://mlsanity.com/paper/2602.15456/) (2026)
- [Towards Effective Discrimination Testing for Generative AI](https://mlsanity.com/paper/2412.21052/) (2024)
- [More Women, Same Stereotypes: Unpacking the Gender Bias Paradox in Large Language Models](https://mlsanity.com/paper/2503.15904/) (2025)
- [Neutralizing the Narrative: AI-Powered Debiasing of Online News Articles](https://mlsanity.com/paper/2504.03520/) (2025)
- [Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias](https://mlsanity.com/paper/2601.18486/) (2026)
- [Investigating Political and Demographic Associations in Large Language Models Through Moral Foundations Theory](https://mlsanity.com/paper/2510.13902/) (2025)
- [EQUITRIAGE: A Fairness Audit of Gender Bias in LLM-Based Emergency Department Triage](https://mlsanity.com/paper/2605.03998/) (2026)
- [AgentFairBench: Do LLM Agents Discriminate When They Act?](https://mlsanity.com/paper/2606.16723/) (2026)
- [The Model Hears You: Audio Language Model Deployments Should Consider the Principle of Least Privilege](https://mlsanity.com/paper/2503.16833/) (2025)
- [A Scalable Entity-Based Framework for Auditing Bias in Large Language Models](https://mlsanity.com/paper/2601.12374/) (2026)
- [Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback](https://mlsanity.com/paper/2603.12471/) (2026)
- [FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocation](https://mlsanity.com/paper/2607.28934/) (2026)
- [Prompt and Prejudice](https://mlsanity.com/paper/2408.04671/) (2024)

## Abstract (arXiv)

We employ an audit design to investigate biases in state-of-the-art large language models, including GPT-4. In our study, we prompt the models for advice involving a named individual across a variety of scenarios, such as during car purchase negotiations or election outcome predictions. We find that the advice systematically disadvantages names that are commonly associated with racial minorities and women. Names associated with Black women receive the least advantageous outcomes. The biases are consistent across 42 prompt templates and several models, indicating a systemic issue rather than isolated incidents. While providing numerical, decision-relevant anchors in the prompt can successfully counteract the biases, qualitative details have inconsistent effects and may even increase disparities. Our findings underscore the importance of conducting audits at the point of LLM deployment and implementation to mitigate their potential for harm against marginalized communities.