# Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education

arXiv: https://arxiv.org/abs/2503.19182 · HTML: https://mlsanity.com/paper/2503.19182/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey](https://mlsanity.com/paper/2309.13933/) (2023)
- [Less is More for Long Document Summary Evaluation by LLMs](https://mlsanity.com/paper/2309.07382/) (2023)
- [Intersectional Bias in Causal Language Models](https://mlsanity.com/paper/2107.07691/) (2021)
- [Are Emily and Greg Still More Employable than Lakisha and Jamal? Investigating Algorithmic Hiring Bias in the Era of ChatGPT](https://mlsanity.com/paper/2310.05135/) (2023)
- [Auditing the Use of Language Models to Guide Hiring Decisions](https://mlsanity.com/paper/2404.03086/) (2024)
- [A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics](https://mlsanity.com/paper/2307.03195/) (2023)
- [Machamp: A Generalized Entity Matching Benchmark](https://mlsanity.com/paper/2106.08455/) (2021)
- [Identification of Bias Against People with Disabilities in Sentiment Analysis and Toxicity Detection Models](https://mlsanity.com/paper/2111.13259/) (2021)
- [Evaluation of LLMs Biases Towards Elite Universities: A Persona-Based Exploration](https://mlsanity.com/paper/2407.12801/) (2024)

## Built on by (most cited first, 14 of 14)

- [Robustly Improving LLM Fairness in Realistic Settings via Interpretability](https://mlsanity.com/paper/2506.10922/) (2025)
- [Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs](https://mlsanity.com/paper/2601.03087/) (2026)
- [Counterfactual Fairness Evaluation of LLM-Based Contact Center Agent Quality Assurance System](https://mlsanity.com/paper/2602.14970/) (2026)
- [PopResume: Causal Fairness Evaluation of LLM/VLM Resume Screeners with Population-Representative Dataset](https://mlsanity.com/paper/2603.22714/) (2026)
- [Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring](https://mlsanity.com/paper/2604.19984/) (2026)
- [MSLEF: Multi-Segment LLM Ensemble Finetuning in Recruitment](https://mlsanity.com/paper/2509.06200/) (2025)
- [Synapse: Evolving Job-Person Fit with Explainable Two-phase Retrieval and LLM-guided Genetic Resume Optimization](https://mlsanity.com/paper/2604.02539/) (2026)
- [Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues](https://mlsanity.com/paper/2606.31644/) (2026)
- [Institutional Prestige as Geographic Bias in Large Language Models: Evidence from Three Factorial Experiments with Bootstrap Confidence Intervals](https://mlsanity.com/paper/2608.18107/) (2026)
- [Beyond Outcome Gaps: Process-Aware Fairness Diagnosis for LLM-based Multi-Agent Decision Systems](https://mlsanity.com/paper/2609.02092/) (2026)
- [Beyond the Name: Demographic Leakage in De-Identified R'esum'es and Evaluation Artifacts in LLM Bias Audits](https://mlsanity.com/paper/2609.16501/) (2026)
- [Competence-Preserving Resume Perturbations Expose Presentation Sensitivity in LLM Screening](https://mlsanity.com/paper/2609.16517/) (2026)
- [When Hiring Becomes Agent-Mediated: Evaluating Access and Recurrence in Two-Agent R'esum'e Screening](https://mlsanity.com/paper/2609.19530/) (2026)
- [EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI](https://mlsanity.com/paper/2610.16689/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) offer the potential to automate hiring by matching job descriptions with candidate resumes, streamlining recruitment processes, and reducing operational costs. However, biases inherent in these models may lead to unfair hiring practices, reinforcing societal prejudices and undermining workplace diversity. This study examines the performance and fairness of LLMs in job-resume matching tasks within the English language and U.S. context. It evaluates how factors such as gender, race, and educational background influence model decisions, providing critical insights into the fairness and reliability of LLMs in HR applications. Our findings indicate that while recent models have reduced biases related to explicit attributes like gender and race, implicit biases concerning educational background remain significant. These results highlight the need for ongoing evaluation and the development of advanced bias mitigation strategies to ensure equitable hiring practices when using LLMs in industry settings.