# Revealing Hidden Bias in AI: Lessons from Large Language Models

arXiv: https://arxiv.org/abs/2410.16927 · HTML: https://mlsanity.com/paper/2410.16927/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 9 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 4 of 4)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Fairness in Machine Learning: Lessons from Political Philosophy](https://mlsanity.com/paper/1712.03586/) (2017)
- [Dbias: Detecting biases and ensuring Fairness in news articles](https://mlsanity.com/paper/2208.05777/) (2022)

## Built on by (most cited first, 2 of 2)

- [Ontology-constrained multi-LLM scoring of hypothesis support in the predictive processing literature](https://mlsanity.com/paper/2606.05206/) (2026)
- [Position: Fairness Failure in Generative Models is an Evaluation Problem](https://mlsanity.com/paper/2608.16974/) (2026)

## Abstract (arXiv)

As large language models (LLMs) become integral to recruitment processes, concerns about AI-induced bias have intensified. This study examines biases in candidate interview reports generated by Claude 3.5 Sonnet, GPT-4o, Gemini 1.5, and Llama 3.1 405B, focusing on characteristics such as gender, race, and age. We evaluate the effectiveness of LLM-based anonymization in reducing these biases. Findings indicate that while anonymization reduces certain biases, particularly gender bias, the degree of effectiveness varies across models and bias types. Notably, Llama 3.1 405B exhibited the lowest overall bias. Moreover, our methodology of comparing anonymized and non-anonymized data reveals a novel approach to assessing inherent biases in LLMs beyond recruitment applications. This study underscores the importance of careful LLM selection and suggests best practices for minimizing bias in AI applications, promoting fairness and inclusivity.