# Structured Reasoning for Fairness: A Multi-Agent Approach to Bias Detection in Textual Data

arXiv: https://arxiv.org/abs/2503.00355 · HTML: https://mlsanity.com/paper/2503.00355/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 1 Mar 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Measuring and Reducing Gendered Correlations in Pre-trained Models](https://mlsanity.com/paper/2010.06032/) (2020)
- [Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.03310/) (2019)
- [On Measuring and Mitigating Biased Inferences of Word Embeddings](https://mlsanity.com/paper/1908.09369/) (2019)
- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [Unsupervised Discovery of Implicit Gender Bias](https://mlsanity.com/paper/2004.08361/) (2020)
- [Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection](https://mlsanity.com/paper/2501.02295/) (2025)
- [Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency](https://mlsanity.com/paper/2402.15481/) (2024)
- [Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models](https://mlsanity.com/paper/2211.08461/) (2022)
- [It is not "accuracy vs. explainability" -- we need both for trustworthy AI systems](https://mlsanity.com/paper/2212.11136/) (2022)
- [Identifying and Measuring Token-Level Sentiment Bias in Pre-trained Language Models with Prompts](https://mlsanity.com/paper/2204.07289/) (2022)
- [Exploring Subjectivity for more Human-Centric Assessment of Social Biases in Large Language Models](https://mlsanity.com/paper/2405.11048/) (2024)

## Built on by (most cited first, 2 of 2)

- [CEA-LIST at CheckThat! 2025: Evaluating LLMs as Detectors of Bias and Opinion in Text](https://mlsanity.com/paper/2507.07539/) (2025)
- [The Wisdom of Agent Crowds: A Human-AI Interaction Innovation Ignition Framework](https://mlsanity.com/paper/2505.06947/) (2025)

## Abstract (arXiv)

From disinformation spread by AI chatbots to AI recommendations that inadvertently reinforce stereotypes, textual bias poses a significant challenge to the trustworthiness of large language models (LLMs). In this paper, we propose a multi-agent framework that systematically identifies biases by disentangling each statement as fact or opinion, assigning a bias intensity score, and providing concise, factual justifications. Evaluated on 1,500 samples from the WikiNPOV dataset, the framework achieves 84.9% accuracyunicodex2014an improvement of 13.0% over the zero-shot baselineunicodex2014demonstrating the efficacy of explicitly modeling fact versus opinion prior to quantifying bias intensity. By combining enhanced detection accuracy with interpretable explanations, this approach sets a foundation for promoting fairness and accountability in modern language models.