# BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs

arXiv: https://arxiv.org/abs/2407.10241 · HTML: https://mlsanity.com/paper/2407.10241/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 14 Jul 2024.

## Builds on (most cited first, 30 of 30)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases](https://mlsanity.com/paper/2006.03955/) (2020)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Reducing Sentiment Bias in Language Models via Counterfactual Evaluation](https://mlsanity.com/paper/1911.03064/) (2019)
- [UnQovering Stereotyping Biases via Underspecified Questions](https://mlsanity.com/paper/2010.02428/) (2020)
- [Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation](https://mlsanity.com/paper/2305.07609/) (2023)
- [Fast Model Debias with Machine Unlearning](https://mlsanity.com/paper/2310.12560/) (2023)
- [BiasAsker: Measuring the Bias in Conversational AI System](https://mlsanity.com/paper/2305.12434/) (2023)
- [Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting](https://mlsanity.com/paper/2401.15585/) (2024)
- [Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks](https://mlsanity.com/paper/2210.02938/) (2022)
- [On the Independence of Association Bias and Empirical Fairness in Language Models](https://mlsanity.com/paper/2304.10153/) (2023)
- [Large Language Model Bias Mitigation from the Perspective of Knowledge Editing](https://mlsanity.com/paper/2405.09341/) (2024)
- [IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators](https://mlsanity.com/paper/2402.00345/) (2024)

## Built on by (most cited first, 21 of 21)

- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [PAD: Personalized Alignment of LLMs at Decoding-Time](https://mlsanity.com/paper/2410.04070/) (2024)
- [FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering](https://mlsanity.com/paper/2504.14492/) (2025)
- [FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs](https://mlsanity.com/paper/2410.19317/) (2024)
- [A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directions](https://mlsanity.com/paper/2409.16430/) (2024)
- [Datasets and Recipes for Video Temporal Grounding via Reinforcement Learning](https://mlsanity.com/paper/2507.18100/) (2025)
- [Identifying and Mitigating Social Bias Knowledge in Language Models](https://mlsanity.com/paper/2408.11843/) (2024)
- [Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment](https://mlsanity.com/paper/2504.12663/) (2025)
- [The Bias is in the Details: An Assessment of Cognitive Bias in LLMs](https://mlsanity.com/paper/2509.22856/) (2025)
- [Unsupervised Concept Vector Extraction for Bias Control in LLMs](https://mlsanity.com/paper/2502.19721/) (2025)
- [BiasFilter: An Inference-Time Debiasing Framework for Large Language Models](https://mlsanity.com/paper/2505.23829/) (2025)
- [DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models](https://mlsanity.com/paper/2503.04240/) (2025)
- [No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models](https://mlsanity.com/paper/2503.11985/) (2025)
- [BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models](https://mlsanity.com/paper/2504.21299/) (2025)
- [Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions](https://mlsanity.com/paper/2509.25539/) (2025)
- [Is General-Purpose AI Reasoning Sensitive to Data-Induced Cognitive Biases? Dynamic Benchmarking on Typical Software Engineering Dilemmas](https://mlsanity.com/paper/2508.11278/) (2025)
- [BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses](https://mlsanity.com/paper/2510.00232/) (2025)
- [Detection, Classification, and Mitigation of Gender Bias in Large Language Models](https://mlsanity.com/paper/2506.12527/) (2025)
- [Cross-Language Bias Examination in Large Language Models](https://mlsanity.com/paper/2512.16029/) (2025)
- [MASteer: Multi-Agent Adaptive Steer Strategy for End-to-End LLM Trustworthiness Repair](https://mlsanity.com/paper/2508.06963/) (2025)

## Abstract (arXiv)

Evaluating the bias in Large Language Models (LLMs) becomes increasingly crucial with their rapid development. However, existing evaluation methods rely on fixed-form outputs and cannot adapt to the flexible open-text generation scenarios of LLMs (e.g., sentence completion and question answering). To address this, we introduce BiasAlert, a plug-and-play tool designed to detect social bias in open-text generations of LLMs. BiasAlert integrates external human knowledge with inherent reasoning capabilities to detect bias reliably. Extensive experiments demonstrate that BiasAlert significantly outperforms existing state-of-the-art methods like GPT4-as-A-Judge in detecting bias. Furthermore, through application studies, we demonstrate the utility of BiasAlert in reliable LLM bias evaluation and bias mitigation across various scenarios. Model and code will be publicly released.