# Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts

arXiv: https://arxiv.org/abs/2501.04316 · HTML: https://mlsanity.com/paper/2501.04316/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 8 Jan 2025.

## Builds on (most cited first, 30 of 31)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Measuring and Reducing Gendered Correlations in Pre-trained Models](https://mlsanity.com/paper/2010.06032/) (2020)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle](https://mlsanity.com/paper/1901.10002/) (2019)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Intrinsic Bias Metrics Do Not Correlate with Application Bias](https://mlsanity.com/paper/2012.15859/) (2020)
- [Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models](https://mlsanity.com/paper/2102.04130/) (2021)
- [What's In My Big Data?](https://mlsanity.com/paper/2310.20707/) (2023)
- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations](https://mlsanity.com/paper/2203.13928/) (2022)
- [Understanding Undesirable Word Embedding Associations](https://mlsanity.com/paper/1908.06361/) (2019)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval](https://mlsanity.com/paper/2407.20371/) (2024)
- [What does it mean to solve the problem of discrimination in hiring? Social, technical and legal perspectives from the UK on automated hiring systems](https://mlsanity.com/paper/1910.06144/) (2019)
- ["You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations](https://mlsanity.com/paper/2406.12232/) (2024)
- [Identifying and Improving Disability Bias in GPT-Based Resume Screening](https://mlsanity.com/paper/2402.01732/) (2024)
- [Are Emily and Greg Still More Employable than Lakisha and Jamal? Investigating Algorithmic Hiring Bias in the Era of ChatGPT](https://mlsanity.com/paper/2310.05135/) (2023)
- [Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLP](https://mlsanity.com/paper/2405.17159/) (2024)
- [Auditing the Use of Language Models to Guide Hiring Decisions](https://mlsanity.com/paper/2404.03086/) (2024)
- [Bias in News Summarization: Measures, Pitfalls and Corpora](https://mlsanity.com/paper/2309.08047/) (2023)
- [Fair Abstractive Summarization of Diverse Perspectives](https://mlsanity.com/paper/2311.07884/) (2023)
- [Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankers](https://mlsanity.com/paper/2404.03192/) (2024)

## Built on by (most cited first, 1 of 1)

- [Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring](https://mlsanity.com/paper/2604.19984/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly being deployed in high-stakes applications like hiring, yet their potential for unfair decision-making remains understudied in generative and retrieval settings. In this work, we examine the allocational fairness of LLM-based hiring systems through two tasks that reflect actual HR usage: resume summarization and applicant ranking. By constructing a synthetic resume dataset with controlled perturbations and curating job postings, we investigate whether model behavior differs across demographic groups. Our findings reveal that generated summaries exhibit meaningful differences more frequently for race than for gender perturbations. Models also display non-uniform retrieval selection patterns across demographic groups and exhibit high ranking sensitivity to both gender and race perturbations. Surprisingly, retrieval models can show comparable sensitivity to both demographic and non-demographic changes, suggesting that fairness issues may stem from broader model brittleness. Overall, our results indicate that LLM-based hiring systems, especially in the retrieval stage, can exhibit notable biases that lead to discriminatory outcomes in real-world contexts.