Investigating Thinking Behaviours of Reasoning-Based Language Models for Social Bias Mitigation
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
While reasoning-based large language models excel at complex tasks through an internal, structured thinking process, a concerning phenomenon has emerged that such a thinking process can aggregate social stereotypes, leading to biased outcomes. However, the underlying behaviours of these language models in social bias scenarios remain underexplored. In this work, we systematically investigate mechanisms within the thinking process behind this phenomenon and uncover two failure patterns that drive social bias aggregation: 1) stereotype repetition, where the model relies on social stereotypes as its primary justification, and 2) irrelevant information injection, where it fabricates or introduces new details to support a biased narrative. Building on these insights, we introduce a lightweight prompt-based mitigation approach that queries the model to review its own initial reasoning against these specific failure patterns. Experiments on question answering (BBQ and StereoSet) and open-ended (BOLD) benchmarks show that our approach effectively reduces bias while maintaining or improving accuracy.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 16 of its 46 references that are in the map. First posted on arXiv 20 Oct 2025.
Builds on
16 of 46 references.
Closest ancestors · 4
Same direction, newest first
- 2025Is Reasoning All You Need? Probing Bias in the Age of Reasoning Language Models3 Jul
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Models24 Feb
- 2025Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning21 Feb
- 2024Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting28 Jan
Other references · 12
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
- 2025Reasoning Models Don't Always Say What They Thinkcited by 415
- 2025Dynamic Early Exit in Reasoning Modelscited by 219
- 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 183
- 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMscited by 117
- 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillationcited by 43
- 2024JBBQ: Japanese Bias Benchmark for Analyzing Social Biases in Large Language Modelscited by 7
- 2025Evaluating the Effect of Retrieval Augmentation on Social Biasescited by 7
- 2025Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?cited by 7
- 2025Intent-Aware Self-Correction for Mitigating Social Biases in Large Language Modelscited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025BiasCause: Evaluate Socially Biased Causal Reasoning of Large Language Models
- 2026Evaluating Social Bias in RAG Systems: When External Context Helps and Reasoning Hurts
- 2026Inference-Time Reasoning Selectively Reduces Implicit Social Bias in Large Language Models
- 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
- 2026Multi-Persona Thinking for Bias Mitigation in Large Language Models
- 2022SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models
- 2023SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models
- 2024Mitigating Social Bias in Large Language Models: A Multi-Objective Approach within a Multi-Agent Framework