# Social Bias Probing: Fairness Benchmarking for Language Models

arXiv: https://arxiv.org/abs/2311.09090 · HTML: https://mlsanity.com/paper/2311.09090/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 71 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 30 of 30)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics](https://mlsanity.com/paper/2106.14574/) (2021)
- [Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents](https://mlsanity.com/paper/2201.04723/) (2022)
- [Unmasking the Mask -- Evaluating Social Biases in Masked Language Models](https://mlsanity.com/paper/2104.07496/) (2021)
- [Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Tasks](https://mlsanity.com/paper/2306.05550/) (2023)
- [It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia](https://mlsanity.com/paper/1501.06307/) (2015)
- [Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Models](https://mlsanity.com/paper/2104.07505/) (2021)
- [Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers](https://mlsanity.com/paper/2210.07362/) (2022)
- [Measuring Gender Bias in West Slavic Language Models](https://mlsanity.com/paper/2304.05783/) (2023)
- [In-Depth Look at Word Filling Societal Bias Measures](https://mlsanity.com/paper/2302.12640/) (2023)
- [Measuring Harmful Representations in Scandinavian Language Models](https://mlsanity.com/paper/2211.11678/) (2022)

## Built on by (most cited first, 30 of 32)

- [SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety](https://mlsanity.com/paper/2404.05399/) (2024)
- [Building Guardrails for Large Language Models](https://mlsanity.com/paper/2402.01822/) (2024)
- [Social Bias Evaluation for Large Language Models Requires Prompt Variations](https://mlsanity.com/paper/2407.03129/) (2024)
- [What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks](https://mlsanity.com/paper/2411.03343/) (2024)
- [Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge](https://mlsanity.com/paper/2504.07887/) (2025)
- [Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias](https://mlsanity.com/paper/2405.15739/) (2024)
- [BiasDora: Exploring Hidden Biased Associations in Vision-Language Models](https://mlsanity.com/paper/2407.02066/) (2024)
- [How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studies](https://mlsanity.com/paper/2407.11733/) (2024)
- [Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation](https://mlsanity.com/paper/2407.08441/) (2024)
- [FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes](https://mlsanity.com/paper/2506.23111/) (2025)
- [What's Not Said Still Hurts: A Description-Based Evaluation Framework for Measuring Social Bias in LLMs](https://mlsanity.com/paper/2502.19749/) (2025)
- [FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Models](https://mlsanity.com/paper/2503.19540/) (2025)
- [Rethinking LLM Bias Probing Using Lessons from the Social Sciences](https://mlsanity.com/paper/2503.00093/) (2025)
- [Revealing Fine-Grained Values and Opinions in Large Language Models](https://mlsanity.com/paper/2406.19238/) (2024)
- [When Agents See Humans as the Outgroup: Belief-Dependent Bias in LLM-Powered Agents](https://mlsanity.com/paper/2601.00240/) (2026)
- [A Group Fairness Lens for Large Language Models](https://mlsanity.com/paper/2312.15478/) (2023)
- [Histoires Morales: A French Dataset for Assessing Moral Alignment](https://mlsanity.com/paper/2501.17117/) (2025)
- [Are Bias Evaluation Methods Biased ?](https://mlsanity.com/paper/2506.17111/) (2025)
- [Veracity Bias and Beyond: Uncovering LLMs' Hidden Beliefs in Problem-Solving Reasoning](https://mlsanity.com/paper/2505.16128/) (2025)
- [DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMs](https://mlsanity.com/paper/2505.10013/) (2025)
- [BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injection](https://mlsanity.com/paper/2508.08855/) (2025)
- [Large Language Models Are Still Misled by Simple Bias Ensembles](https://mlsanity.com/paper/2505.16522/) (2025)
- [Do They Understand Them? An Updated Evaluation on Nonbinary Pronoun Handling in Large Language Models](https://mlsanity.com/paper/2508.00788/) (2025)
- [Fair-GPTQ: Bias-Aware Quantization for Large Language Models](https://mlsanity.com/paper/2509.15206/) (2025)
- [Debiasing LLMs by Masking Unfairness-Driving Attention Heads](https://mlsanity.com/paper/2510.10142/) (2025)
- [Uncovering Competency Gaps in Large Language Models and Their Benchmarks](https://mlsanity.com/paper/2512.20638/) (2025)
- [Analysing LLM Persona Generation and Fairness Interpretation in Polarised Geopolitical Contexts](https://mlsanity.com/paper/2603.22837/) (2026)
- [Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement](https://mlsanity.com/paper/2606.17506/) (2026)
- [The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data](https://mlsanity.com/paper/2608.04268/) (2026)
- [Personalization Without Stereotyping: Do Speech LLMs Know When to Turn a Deaf Ear?](https://mlsanity.com/paper/2610.13550/) (2026)

## Abstract (arXiv)

While the impact of social biases in language models has been recognized, prior methods for bias evaluation have been limited to binary association tests on small datasets, limiting our understanding of bias complexities. This paper proposes a novel framework for probing language models for social biases by assessing disparate treatment, which involves treating individuals differently according to their affiliation with a sensitive demographic group. We curate SoFa, a large-scale benchmark designed to address the limitations of existing fairness collections. SoFa expands the analysis beyond the binary comparison of stereotypical versus anti-stereotypical identities to include a diverse range of identities and stereotypes. Comparing our methodology with existing benchmarks, we reveal that biases within language models are more nuanced than acknowledged, indicating a broader scope of encoded biases than previously recognized. Benchmarking LMs on SoFa, we expose how identities expressing different religions lead to the most pronounced disparate treatments across all models. Finally, our findings indicate that real-life adversities faced by various groups such as women and people with disabilities are mirrored in the behavior of these models.