# Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systems

arXiv: https://arxiv.org/abs/2409.19804 · HTML: https://mlsanity.com/paper/2409.19804/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 29 Sep 2024.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 26 of 26)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research](https://mlsanity.com/paper/2405.13576/) (2024)
- [UnQovering Stereotyping Biases via Underspecified Questions](https://mlsanity.com/paper/2010.02428/) (2020)
- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering](https://mlsanity.com/paper/2404.04302/) (2024)
- [SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs](https://mlsanity.com/paper/2404.13081/) (2024)
- [Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers](https://mlsanity.com/paper/2104.13640/) (2021)
- [FairRAG: Fair Human Generation via Fair Retrieval Augmentation](https://mlsanity.com/paper/2403.19964/) (2024)
- [Do Neural Ranking Models Intensify Gender Bias?](https://mlsanity.com/paper/2005.00372/) (2020)
- [STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answering](https://mlsanity.com/paper/2406.19973/) (2024)
- [Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankers](https://mlsanity.com/paper/2404.03192/) (2024)
- [Overview of the TREC 2022 Fair Ranking Track](https://mlsanity.com/paper/2302.05558/) (2023)

## Built on by (most cited first, 26 of 26)

- [Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities](https://mlsanity.com/paper/2505.20099/) (2025)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.14891/) (2025)
- [No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users](https://mlsanity.com/paper/2410.07589/) (2024)
- [Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs](https://mlsanity.com/paper/2506.11415/) (2025)
- [Evaluating the Effect of Retrieval Augmentation on Social Biases](https://mlsanity.com/paper/2502.17611/) (2025)
- [Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets](https://mlsanity.com/paper/2504.20119/) (2025)
- [Mitigating Bias in RAG: Controlling the Embedder](https://mlsanity.com/paper/2502.17390/) (2025)
- [Bias-Aware Agent: Enhancing Fairness in AI-Driven Knowledge Retrieval](https://mlsanity.com/paper/2503.21237/) (2025)
- [EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.13506/) (2025)
- [SafeSearch: Automated Red-Teaming of LLM-Based Search Agents](https://mlsanity.com/paper/2509.23694/) (2025)
- [RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contexts](https://mlsanity.com/paper/2510.05310/) (2025)
- [Safety Degradation in AI Agents](https://mlsanity.com/paper/2505.14215/) (2025)
- [Gender Encoding Patterns in Pretrained Language Model Representations](https://mlsanity.com/paper/2503.06734/) (2025)
- [Evaluating Social Bias in RAG Systems: When External Context Helps and Reasoning Hurts](https://mlsanity.com/paper/2602.09442/) (2026)
- [The Impact of Ideological Discourses in RAG: A Case Study with COVID-19 Treatments](https://mlsanity.com/paper/2603.14838/) (2026)
- [Who Benefits from RAG? The Role of Exposure, Utility and Attribution Bias](https://mlsanity.com/paper/2603.24218/) (2026)
- [DiscourseFlip: An Oblique Discourse-Level Opinion Manipulation Attack against Black-box Retrieval-Augmented Generation](https://mlsanity.com/paper/2606.01212/) (2026)
- [Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.24977/) (2026)
- [The Other Side of the Coin: Exploring Fairness in Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.12323/) (2025)
- [Enhancing Critical Thinking with AI: A Tailored Warning System for RAG Models](https://mlsanity.com/paper/2504.16883/) (2025)
- [Quantifying Document Impact in RAG-LLMs](https://mlsanity.com/paper/2601.05260/) (2025)
- [Fairness-Aware Retrieval Optimization for Retrieval-Augmented Generation](https://mlsanity.com/paper/2605.15790/) (2026)
- [Towards FairRAG: Preventing Representational Harm in Retrieval-Augmented Generation by Enforcing Fair Exposure at Retrieval Time](https://mlsanity.com/paper/2605.18806/) (2026)
- [Building Trustworthy Graph-Agentic RAG for Social Good: Architectures, Failure Propagation, and Assurance by Construction](https://mlsanity.com/paper/2609.06391/) (2026)
- [Retrieval Sensitivity to Identity Signals in Queries](https://mlsanity.com/paper/2609.36534/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has recently gained significant attention for its enhanced ability to integrate external knowledge sources into open-domain question answering (QA) tasks. However, it remains unclear how these models address fairness concerns, particularly with respect to sensitive attributes such as gender, geographic location, and other demographic factors. First, as language models evolve to prioritize utility, like improving exact match accuracy, fairness considerations may have been largely overlooked. Second, the complex, multi-component architecture of RAG methods poses challenges in identifying and mitigating biases, as each component is optimized for distinct objectives. In this paper, we aim to empirically evaluate fairness in several RAG methods. We propose a fairness evaluation framework tailored to RAG, using scenario-based questions and analyzing disparities across demographic attributes. Our experimental results indicate that, despite recent advances in utility-driven optimization, fairness issues persist in both the retrieval and generation stages. These findings underscore the need for targeted interventions to address fairness concerns throughout the RAG pipeline. The dataset and code used in this study are publicly available at this GitHub Repository https://github.com/elviswxy/RAG_fairness .