mlsanityData to 14 Oct 2026

Do Biased Models Have Biased Thoughts?

arXiv 2508.06671 ↗First posted 8 Aug 2025cs.CL cs.AIcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The impressive performance of language models is undeniable. However, the presence of biases based on gender, race, socio-economic status, physical appearance, and sexual orientation makes the deployment of language models challenging. This paper studies the effect of chain-of-thought prompting, a recent approach that studies the steps followed by the model before it responds, on fairness. More specifically, we ask the following question: textitDo biased models have biased thoughts? To answer our question, we conduct experiments on 5 popular large language models using fairness metrics to quantify 11 different biases in the model's thoughts and output. Our results show that the bias in the thinking steps is not highly correlated with the output bias (less than 0.6 correlation with a p-value smaller than 0.001 in most cases). In other words, unlike human beings, the tested models with biased decisions do not always possess biased thoughts.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 17 of its 68 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

17 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Improve LLM-as-a-Judge Ability as a General Ability17 Feb

Other references · 16

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2023Mistral 7Bcited by 3,164
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  7. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
  10. 2024Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Modelscited by 33
  11. 2024Dissociation of Faithful and Unfaithful Reasoning in LLMscited by 23
  12. 2024First-Person Fairness in Chatbotscited by 20
4 more
  1. 2024Chain of Logic: Rule-Based Reasoning with Large Language Modelscited by 18
  2. 2023Should We Attend More or Less? Modulating Attention for Fairnesscited by 11
  3. 2024Data Generation Using Large Language Models for Text Classification: An Empirical Case Studycited by 11
  4. 2024Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Modelscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Do LLMs Know a Good Hypothesis When They See One? Logit-Based Energy Scoring Outperforms Prompted LLM-as-Judge for Scientific Hypothesis Rankingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting
  2. 2026Mechanics of Bias and Reasoning: Interpreting the Impact of Chain-of-Thought Prompting on Gender Bias in LLMs
  3. 2025Debiasing LLMs by Masking Unfairness-Driving Attention Heads
  4. 2025Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection
  5. 2025Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models
  6. 2026Does Thinking Help Fairness? Reasoning Tokens Resolve Some Biases but Create More
  7. 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
  8. 2025Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution