# Intersectional Fairness in Large Language Models

arXiv: https://arxiv.org/abs/2604.20677 · HTML: https://mlsanity.com/paper/2604.20677/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 60 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Builds on (most cited first, 30 of 30)

- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [An Intersectional Definition of Fairness](https://mlsanity.com/paper/1807.08362/) (2018)
- [Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity](https://mlsanity.com/paper/2301.12867/) (2023)
- [KoBBQ: Korean Bias Benchmark for Question Answering](https://mlsanity.com/paper/2307.16778/) (2023)
- [Fairness Testing: A Comprehensive Survey and Analysis of Trends](https://mlsanity.com/paper/2207.10223/) (2022)
- [Socially Responsible AI Algorithms: Issues, Purposes, and Challenges](https://mlsanity.com/paper/2101.02032/) (2021)
- [BiasAsker: Measuring the Bias in Conversational AI System](https://mlsanity.com/paper/2305.12434/) (2023)
- [Social Bias Evaluation for Large Language Models Requires Prompt Variations](https://mlsanity.com/paper/2407.03129/) (2024)
- [Exploring the Impact of Temperature on Large Language Models:Hot or Cold?](https://mlsanity.com/paper/2506.07295/) (2025)
- [Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answering](https://mlsanity.com/paper/2503.14996/) (2025)
- [CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models](https://mlsanity.com/paper/2306.16244/) (2023)
- [Social Bias Benchmark for Generation: A Comparison of Generation and QA-Based Evaluations](https://mlsanity.com/paper/2503.06987/) (2025)
- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings](https://mlsanity.com/paper/2412.06134/) (2024)
- [Towards a Roadmap on Software Engineering for Responsible AI](https://mlsanity.com/paper/2203.08594/) (2022)
- [BharatBBQ: A Multilingual Bias Benchmark for Question Answering in the Indian Context](https://mlsanity.com/paper/2508.07090/) (2025)
- [Exposing Algorithmic Discrimination and Its Consequences in Modern Society: Insights from a Scoping Study](https://mlsanity.com/paper/2312.04832/) (2023)
- [A Catalog of Fairness-Aware Practices in Machine Learning Engineering](https://mlsanity.com/paper/2408.16683/) (2024)
- [Say It Another Way: Auditing LLMs with a User-Grounded Automated Paraphrasing Framework](https://mlsanity.com/paper/2505.03563/) (2025)
- [Mental Health Equity in LLMs: Leveraging Multi-Hop Question Answering to Detect Amplified and Silenced Perspectives](https://mlsanity.com/paper/2506.18116/) (2025)
- [Towards Systematic Specification and Verification of Fairness Requirements: A Position Paper](https://mlsanity.com/paper/2509.20387/) (2025)
- [Practitioner Insights on Fairness Requirements in the AI Development Life Cycle: An Interview Study](https://mlsanity.com/paper/2512.13830/) (2025)
- [The Perspective of Software Professionals on Algorithmic Racism](https://mlsanity.com/paper/2306.15133/) (2023)
- [Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Development](https://mlsanity.com/paper/2403.15481/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed in socially sensitive settings, raising concerns about fairness and bias, particularly when multiple sensitive attributes intersect. We systematically evaluate intersectional fairness in six LLMs using two datasets from the Bias Benchmark for Question Answering (BBQ), combining race with gender and socioeconomic status. We assess bias, subgroup fairness, accuracy, and consistency across contexts and question polarities. While models perform well in ambiguous contexts, sparse non-unknown predictions limit fairness evaluation. In disambiguated contexts, stereotype alignment affects accuracy differently across datasets: models favor stereotype-reinforcing items in Race-SES but counter-stereotype items in Race-Gender for five of six models. Subgroup fairness metrics reveal uneven outcome distributions despite low observed disparities in some cases, while repeated runs show variable response consistency. Overall, no model consistently achieves both fairness and reliability, highlighting the need to evaluate bias, subgroup fairness, and consistency across intersectional contexts.