# Are Emily and Greg Still More Employable than Lakisha and Jamal? Investigating Algorithmic Hiring Bias in the Era of ChatGPT

arXiv: https://arxiv.org/abs/2310.05135 · HTML: https://mlsanity.com/paper/2310.05135/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [Deep Learning for Procedural Content Generation](https://mlsanity.com/paper/2010.04548/) (2020)
- [JobBERT: Understanding Job Titles through Skills](https://mlsanity.com/paper/2109.09605/) (2021)
- [Learning to Match Jobs with Resumes from Sparse Interaction Data using Multi-View Co-Teaching Network](https://mlsanity.com/paper/2009.13299/) (2020)
- [Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programming](https://mlsanity.com/paper/2111.07875/) (2021)
- [Surfacing Biases in Large Language Models using Contrastive Input Decoding](https://mlsanity.com/paper/2305.07378/) (2023)
- [Domain Adaptation for Resume Classification Using Convolutional Neural Networks](https://mlsanity.com/paper/1707.05576/) (2017)

## Built on by (most cited first, 24 of 24)

- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation](https://mlsanity.com/paper/2402.12649/) (2024)
- [The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring](https://mlsanity.com/paper/2405.04412/) (2024)
- [Identifying and Improving Disability Bias in GPT-Based Resume Screening](https://mlsanity.com/paper/2402.01732/) (2024)
- [Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?](https://mlsanity.com/paper/2406.10486/) (2024)
- [Auditing the Use of Language Models to Guide Hiring Decisions](https://mlsanity.com/paper/2404.03086/) (2024)
- [Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education](https://mlsanity.com/paper/2503.19182/) (2025)
- [Social Debiasing for Fair Multi-modal LLMs](https://mlsanity.com/paper/2408.06569/) (2024)
- [Rethinking LLM Bias Probing Using Lessons from the Social Sciences](https://mlsanity.com/paper/2503.00093/) (2025)
- [ABLEIST: Intersectional Disability Bias in LLM-Generated Hiring Scenarios](https://mlsanity.com/paper/2510.10998/) (2025)
- [Can LLMs Hire Fairly? Racial Bias in Resume Screening](https://mlsanity.com/paper/2606.28978/) (2026)
- [Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals](https://mlsanity.com/paper/2404.13885/) (2024)
- [Understanding Gender Bias in AI-Generated Product Descriptions](https://mlsanity.com/paper/2506.05390/) (2025)
- [From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models](https://mlsanity.com/paper/2312.17256/) (2023)
- [Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts](https://mlsanity.com/paper/2501.04316/) (2025)
- [Quantifying Gender Bias in Large Language Models: When ChatGPT Becomes a Hiring Manager](https://mlsanity.com/paper/2604.00011/) (2026)
- [Cheap Learning: Maximising Performance of Language Models for Social Data Science Using Minimal Data](https://mlsanity.com/paper/2401.12295/) (2024)
- [Name of Thrones: Evaluating How LLMs Rank Student Names, Race, and Gender in Status Hierarchies](https://mlsanity.com/paper/2504.10797/) (2025)
- [From job titles to jawlines: Using context voids to study generative AI systems](https://mlsanity.com/paper/2504.13947/) (2025)
- [AutoScreen-FW: An LLM-based Framework for Resume Screening](https://mlsanity.com/paper/2603.18390/) (2026)
- [Decisive: Guiding User Decisions with Optimal Preference Elicitation from Unstructured Documents](https://mlsanity.com/paper/2604.18122/) (2026)
- [Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues](https://mlsanity.com/paper/2606.31644/) (2026)
- [Thin Evidence, Thick Priors: How Language Models Substitute Identity for Missing Financial Facts](https://mlsanity.com/paper/2610.07798/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) such as GPT-3.5, Bard, and Claude exhibit applicability across numerous tasks. One domain of interest is their use in algorithmic hiring, specifically in matching resumes with job categories. Yet, this introduces issues of bias on protected attributes like gender, race and maternity status. The seminal work of Bertrand & Mullainathan (2003) set the gold-standard for identifying hiring bias via field experiments where the response rate for identical resumes that differ only in protected attributes, e.g., racially suggestive names such as Emily or Lakisha, is compared. We replicate this experiment on state-of-art LLMs (GPT-3.5, Bard, Claude and Llama) to evaluate bias (or lack thereof) on gender, race, maternity status, pregnancy status, and political affiliation. We evaluate LLMs on two tasks: (1) matching resumes to job categories; and (2) summarizing resumes with employment relevant information. Overall, LLMs are robust across race and gender. They differ in their performance on pregnancy status and political affiliation. We use contrastive input decoding on open-source LLMs to uncover potential sources of bias.