# Towards Auditing Large Language Models: Improving Text-based Stereotype Detection

arXiv: https://arxiv.org/abs/2311.14126 · HTML: https://mlsanity.com/paper/2311.14126/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate](https://mlsanity.com/paper/1809.02208/) (2018)
- [Multi-Dimensional Gender Bias Classification](https://mlsanity.com/paper/2005.00614/) (2020)
- [SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models](https://mlsanity.com/paper/2305.11840/) (2023)
- [Dbias: Detecting biases and ensuring Fairness in news articles](https://mlsanity.com/paper/2208.05777/) (2022)
- [Intersectional Bias in Causal Language Models](https://mlsanity.com/paper/2107.07691/) (2021)
- [Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs](https://mlsanity.com/paper/2207.00551/) (2022)

## Built on by (most cited first, 13 of 13)

- [GenderAlign: An Alignment Dataset for Mitigating Gender Bias in Large Language Models](https://mlsanity.com/paper/2406.13925/) (2024)
- [Bias Amplification: Large Language Models as Increasingly Biased Media](https://mlsanity.com/paper/2410.15234/) (2024)
- [Bring Your Own Prompts: Use-Case-Specific Bias and Fairness Evaluation for LLMs](https://mlsanity.com/paper/2407.10853/) (2024)
- [SAGED: A Holistic Bias-Benchmarking Pipeline for Language Models with Customisable Fairness Calibration](https://mlsanity.com/paper/2409.11149/) (2024)
- [HEARTS: A Holistic Framework for Explainable, Sustainable and Robust Text Stereotype Detection](https://mlsanity.com/paper/2409.11579/) (2024)
- [LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases](https://mlsanity.com/paper/2501.03112/) (2025)
- [Semantic and Structural Analysis of Implicit Biases in Large Language Models: An Interpretable Approach](https://mlsanity.com/paper/2508.06155/) (2025)
- [Assessing Bias in Metric Models for LLM Open-Ended Generation Bias Benchmarks](https://mlsanity.com/paper/2410.11059/) (2024)
- [Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applications](https://mlsanity.com/paper/2502.00808/) (2025)
- [StereoDetect: Detecting Stereotypes and Anti-stereotypes the Correct Way Using Social Psychological Underpinnings](https://mlsanity.com/paper/2504.03352/) (2025)
- [CALM: Culturally Self-Aware Language Models](https://mlsanity.com/paper/2601.03483/) (2026)
- [Visual Stereotypes of Autism Spectrum in Janus-Pro-7B, DALL-E, Stable Diffusion, SDXL, FLUX, and Midjourney](https://mlsanity.com/paper/2407.16292/) (2024)
- [StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs](https://mlsanity.com/paper/2605.10442/) (2026)

## Abstract (arXiv)

Large Language Models (LLM) have made significant advances in the recent past becoming more mainstream in Artificial Intelligence (AI) enabled human-facing applications. However, LLMs often generate stereotypical output inherited from historical data, amplifying societal biases and raising ethical concerns. This work introduces i) the Multi-Grain Stereotype Dataset, which includes 52,751 instances of gender, race, profession and religion stereotypic text and ii) a novel stereotype classifier for English text. We design several experiments to rigorously test the proposed model trained on the novel dataset. Our experiments show that training the model in a multi-class setting can outperform the one-vs-all binary counterpart. Consistent feature importance signals from different eXplainable AI tools demonstrate that the new model exploits relevant text features. We utilise the newly created model to assess the stereotypic behaviour of the popular GPT family of models and observe the reduction of bias over time. In summary, our work establishes a robust and practical framework for auditing and evaluating the stereotypic bias in LLM.