Irrelevant Alternatives Bias Large Language Model Hiring Decisions
Not on the map. Why?
Abstract (arXiv)
We investigate whether LLMs display a well-known human cognitive bias, the attraction effect, in hiring decisions. The attraction effect occurs when the presence of an inferior candidate makes a superior candidate more appealing, increasing the likelihood of the superior candidate being chosen over a non-dominated competitor. Our study finds consistent and significant evidence of the attraction effect in GPT-3.5 and GPT-4 when they assume the role of a recruiter. Irrelevant attributes of the decoy, such as its gender, further amplify the observed bias. GPT-4 exhibits greater bias variation than GPT-3.5. Our findings remain robust even when warnings against the decoy effect are included and the recruiter role definition is varied.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 11 of its 45 references that are in the map. First posted on arXiv 4 Sep 2024.
Builds on
11 of 45 references.
References · 11
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
- 2022Using cognitive psychology to understand GPT-3cited by 389
- 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
- 2022Thinking Fast and Slow in Large Language Modelscited by 136
- 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Biascited by 51
- 2023Exploring the Sensitivity of LLMs' Decision-Making Capabilities: Insights from Prompt Variation and Hyperparameterscited by 32
- 2024Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpuscited by 32
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.