# Bring Your Own Prompts: Use-Case-Specific Bias and Fairness Evaluation for LLMs

arXiv: https://arxiv.org/abs/2407.10853 · HTML: https://mlsanity.com/paper/2407.10853/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 27 of 27)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Reducing Sentiment Bias in Language Models via Counterfactual Evaluation](https://mlsanity.com/paper/1911.03064/) (2019)
- [Intrinsic Bias Metrics Do Not Correlate with Application Bias](https://mlsanity.com/paper/2012.15859/) (2020)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Fairness in Recommendation Ranking through Pairwise Comparisons](https://mlsanity.com/paper/1903.00780/) (2019)
- [UnQovering Stereotyping Biases via Underspecified Questions](https://mlsanity.com/paper/2010.02428/) (2020)
- [Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation](https://mlsanity.com/paper/2305.07609/) (2023)
- [A Survey on the Fairness of Recommender Systems](https://mlsanity.com/paper/2206.03761/) (2022)
- [Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation](https://mlsanity.com/paper/2109.03858/) (2021)
- [The Impossibility of Fair LLMs](https://mlsanity.com/paper/2406.03198/) (2024)
- [An Audit of Misinformation Filter Bubbles on YouTube: Bubble Bursting and Recent Behavior Changes](https://mlsanity.com/paper/2203.13769/) (2022)
- [Towards Auditing Large Language Models: Improving Text-based Stereotype Detection](https://mlsanity.com/paper/2311.14126/) (2023)
- [A Comprehensive View of the Biases of Toxicity and Sentiment Analysis Methods Towards Utterances with African American English Expressions](https://mlsanity.com/paper/2401.12720/) (2024)

## Built on by (most cited first, 9 of 9)

- [BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Models](https://mlsanity.com/paper/2510.00307/) (2025)
- [Silenced Biases: The Dark Side LLMs Learned to Refuse](https://mlsanity.com/paper/2511.03369/) (2025)
- [SAGED: A Holistic Bias-Benchmarking Pipeline for Language Models with Customisable Fairness Calibration](https://mlsanity.com/paper/2409.11149/) (2024)
- [LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases](https://mlsanity.com/paper/2501.03112/) (2025)
- [Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspective](https://mlsanity.com/paper/2506.19028/) (2025)
- [Uncovering Representation Bias for Investment Decisions in Open-Source Large Language Models](https://mlsanity.com/paper/2510.05702/) (2025)
- [Fairness Evaluation of Large Language Models in Academic Library Reference Services](https://mlsanity.com/paper/2507.04224/) (2025)
- [StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs](https://mlsanity.com/paper/2605.10442/) (2026)
- [Need or Reputation? Region, National Income, and Academic Status in LLM Gatekeeping Decisions](https://mlsanity.com/paper/2608.05178/) (2026)

## Abstract (arXiv)

Bias and fairness risks in Large Language Models (LLMs) vary substantially across deployment contexts, yet existing approaches lack systematic guidance for selecting appropriate evaluation metrics. We present a decision framework that maps LLM use cases, characterized by a model and population of prompts, to relevant bias and fairness metrics based on task type, whether prompts contain protected attribute mentions, and stakeholder priorities. Our framework addresses toxicity, stereotyping, counterfactual unfairness, and allocational harms, and introduces novel metrics based on stereotype classifiers and counterfactual adaptations of text similarity measures. We release an open-source Python library, textttlangfair, for practical adoption. Extensive experiments on use cases across five LLMs and five prompt populations demonstrate that fairness risks cannot be reliably assessed from benchmark performance alone: results on one prompt dataset likely overstate or understate risks for another, underscoring that fairness evaluation must be grounded in the specific deployment context.