# Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankers

arXiv: https://arxiv.org/abs/2404.03192 · HTML: https://mlsanity.com/paper/2404.03192/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 53 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Improving Passage Retrieval with Zero-Shot Question Generation](https://mlsanity.com/paper/2204.07496/) (2022)
- [Fairness in Recommendation Ranking through Pairwise Comparisons](https://mlsanity.com/paper/1903.00780/) (2019)
- [Equity of Attention: Amortizing Individual Fairness in Rankings](https://mlsanity.com/paper/1805.01788/) (2018)
- [Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation](https://mlsanity.com/paper/2305.07609/) (2023)
- [FA*IR: A Fair Top-k Ranking Algorithm](https://mlsanity.com/paper/1706.06368/) (2017)
- [NLPositionality: Characterizing Design Biases of Datasets and Models](https://mlsanity.com/paper/2306.01943/) (2023)
- [Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels](https://mlsanity.com/paper/2310.14122/) (2023)
- [Measuring Fairness in Ranked Outputs](https://mlsanity.com/paper/1610.08559/) (2016)
- [The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models](https://mlsanity.com/paper/2101.05667/) (2021)
- [Ranking with Fairness Constraints](https://mlsanity.com/paper/1704.06840/) (2017)
- [iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making](https://mlsanity.com/paper/1806.01059/) (2018)
- [Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking](https://mlsanity.com/paper/2310.13243/) (2023)

## Built on by (most cited first, 17 of 17)

- [Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown](https://mlsanity.com/paper/2410.00847/) (2024)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2409.19804/) (2024)
- [The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs](https://mlsanity.com/paper/2506.11094/) (2025)
- [Automatically Improving LLM-based Verilog Generation using EDA Tool Feedback](https://mlsanity.com/paper/2411.11856/) (2024)
- [Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)](https://mlsanity.com/paper/2505.21091/) (2025)
- [Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts](https://mlsanity.com/paper/2406.17974/) (2024)
- [How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Models](https://mlsanity.com/paper/2508.16757/) (2025)
- [MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models](https://mlsanity.com/paper/2604.17730/) (2026)
- [Passage-specific Prompt Tuning for Passage Reranking in Question Answering with Large Language Models](https://mlsanity.com/paper/2405.20654/) (2024)
- [ReGeS: Reciprocal Retrieval-Generation Synergy for Conversational Recommender Systems](https://mlsanity.com/paper/2509.21371/) (2025)
- [Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts](https://mlsanity.com/paper/2501.04316/) (2025)
- [An Efficient Long-Context Ranking Architecture With Calibrated LLM Distillation: Application to Person-Job Fit](https://mlsanity.com/paper/2601.10321/) (2026)
- [Not All RAGs Are Created Equal: A Component-Wise Empirical Study for Software Engineering Tasks](https://mlsanity.com/paper/2605.14503/) (2026)
- [Quantifying Misattribution Unfairness in Authorship Attribution](https://mlsanity.com/paper/2506.02321/) (2025)
- [Chitchat with AI: Understand the supply chain carbon disclosure of companies worldwide through Large Language Model](https://mlsanity.com/paper/2511.00024/) (2025)
- [Gender Bias in Emotion Recognition by Large Language Models](https://mlsanity.com/paper/2511.19785/) (2025)

## Abstract (arXiv)

The integration of Large Language Models (LLMs) in information retrieval has raised a critical reevaluation of fairness in the text-ranking models. LLMs, such as GPT models and Llama2, have shown effectiveness in natural language understanding tasks, and prior works (e.g., RankGPT) have also demonstrated that the LLMs exhibit better performance than the traditional ranking models in the ranking task. However, their fairness remains largely unexplored. This paper presents an empirical study evaluating these LLMs using the TREC Fair Ranking dataset, focusing on the representation of binary protected attributes such as gender and geographic location, which are historically underrepresented in search outcomes. Our analysis delves into how these LLMs handle queries and documents related to these attributes, aiming to uncover biases in their ranking algorithms. We assess fairness from both user and content perspectives, contributing an empirical benchmark for evaluating LLMs as the fair ranker.