mlsanityData to 7 Oct 2026

What's in a Name? Auditing Large Language Models for Race and Gender Bias

arXiv 2402.14875 ↗First posted 21 Feb 2024cs.CL cs.AI cs.CYcited by 50

Not on the map. Why?

Abstract (arXiv)

We employ an audit design to investigate biases in state-of-the-art large language models, including GPT-4. In our study, we prompt the models for advice involving a named individual across a variety of scenarios, such as during car purchase negotiations or election outcome predictions. We find that the advice systematically disadvantages names that are commonly associated with racial minorities and women. Names associated with Black women receive the least advantageous outcomes. The biases are consistent across 42 prompt templates and several models, indicating a systemic issue rather than isolated incidents. While providing numerical, decision-relevant anchors in the prompt can successfully counteract the biases, qualitative details have inconsistent effects and may even increase disparities. Our findings underscore the importance of conducting audits at the point of LLM deployment and implementation to mitigate their potential for harm against marginalized communities.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 11 of its 33 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

11 of 33 references.

References · 11

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  2. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  3. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  4. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  5. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  6. 2018The Measure and Mismeasure of Fairnesscited by 94
  7. 2020An Empirical Characterization of Fair Machine Learning For Clinical Risk Predictioncited by 45
  8. 2020Algorithmic Fairness in Educationcited by 44
  9. 2023Are Emily and Greg Still More Employable than Lakisha and Jamal? Investigating Algorithmic Hiring Bias in the Era of ChatGPTcited by 23
  10. 2021Algorithmic Auditing and Social Justice: Lessons from the History of Audit Studiescited by 22
  11. 2020Affirmative Algorithms: The Legal Grounds for Fairness as Awarenesscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 21; 2026: 16.

50 papers.

Sort:

By area:

  1. 2024"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendationscited by 33
  2. 2024LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Educationcited by 28
  3. 2024Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?cited by 24
  4. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  5. 2025Presumed Cultural Identity: How Names Shape LLM Responsescited by 21
  6. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20
  7. 2024Auditing the Use of Language Models to Guide Hiring Decisionscited by 19
  8. 2025Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Educationcited by 13
  9. 2025Detecting Prefix Bias in LLM-based Reward Modelscited by 11
  10. 2024Representation Bias in Political Sample Simulations with Large Language Modelscited by 10
  11. 2024Fairness Testing of Large Language Models in Role-Playingcited by 10
  12. 2024Understanding Intrinsic Socioeconomic Biases in Large Language Modelscited by 9
38 more
  1. 2025DAIQ: Auditing Demographic Attribute Inference from Question in LLMscited by 9
  2. 2025From Superficial Outputs to Superficial Learning: Risks of Large Language Models in Educationcited by 9
  3. 2025All You Need is One: Capsule Prompt Tuning with a Single Vectorcited by 8
  4. 2024Improving LLM Group Fairness on Tabular Data via In-Context Learningcited by 6
  5. 2025More or Less Wrong: A Benchmark for Directional Bias in LLM Comparative Reasoningcited by 6
  6. 2024Towards Effective Discrimination Testing for Generative AIcited by 5
  7. 2025More Women, Same Stereotypes: Unpacking the Gender Bias Paradox in Large Language Modelscited by 5
  8. 2025Neutralizing the Narrative: AI-Powered Debiasing of Online News Articlescited by 4
  9. 2025Investigating Political and Demographic Associations in Large Language Models Through Moral Foundations Theorycited by 3
  10. 2026Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Biascited by 3
  11. 2026In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generationscited by 3
  12. 2026EQUITRIAGE: A Fairness Audit of Gender Bias in LLM-Based Emergency Department Triagecited by 3
  13. 2025The Model Hears You: Audio Language Model Deployments Should Consider the Principle of Least Privilegecited by 2
  14. 2026A Scalable Entity-Based Framework for Auditing Bias in Large Language Modelscited by 2
  15. 2026Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedbackcited by 2
  16. 2026AgentFairBench: Do LLM Agents Discriminate When They Act?cited by 2
  17. 2026FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocationcited by 2
  18. 2024Prompt and Prejudicecited by 1
  19. 2024Implicit Priors Editing in Stable Diffusion via Targeted Token Adjustmentcited by 1
  20. 2025Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contextscited by 1
  21. 2025A Cross-Cultural Comparison of LLM-based Public Opinion Simulation: Evaluating Chinese and U.S. Models on Diverse Societiescited by 1
  22. 2025Yet another algorithmic bias: A Discursive Analysis of Large Language Models Reinforcing Dominant Discourses on Gender and Racecited by 1
  23. 2025Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzlescited by 1
  24. 2026What LLMs Think When You Don't Tell Them What to Think About?cited by 1
  25. 2026Implicit Grading Bias in Large Language Models: How Writing Style Affects Automated Assessment Across Math, Programming, and Essay Taskscited by 1
  26. 2024A Social Outcomes and Priorities centered (SOP) Framework for AI policycited by 0
  27. 2025How Robust Is Homogeneity Bias in LLMs? Evidence Across Models, Decoding Settings, and Identity Signalscited by 0
  28. 2025Token-Level Entropy Reveals Demographic Disparities in Large Language Modelscited by 0
  29. 2025The Biased Samaritan: LLM biases in Perceived Kindnesscited by 0
  30. 2025Fairness Is Not Enough: Auditing Competence and Intersectional Bias in AI-powered Resume Screeningcited by 0
  31. 2025Clustering Discourses: Racial Biases in Short Stories about Women Generated by Large Language Modelscited by 0
  32. 2026White-Box Sensitivity Auditing with Steering Vectorscited by 0
  33. 2026Evaluating the Usage of African-American Vernacular English in Large Language Modelscited by 0
  34. 2026The Geography of Algorithmic Judgment: LLM Intermediaries, Place Identity, and Racial Steering in Housing Searchcited by 0
  35. 2026Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Researchcited by 0
  36. 2026Critical Acclaim Orientation in Large Language Models: Evidence from Film Preference Elicitationcited by 0
  37. 2026Identifying Harm in Personalized, Generative AI Systems Requires User-Centered Auditing at the Interaction Levelcited by 0
  38. 2026The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Auditscited by 0