mlsanityData to 14 Oct 2026

Who Do Language Models Think Is Competent? A Mechanistic Analysis of Occupational Bias

arXiv 2608.20347 ↗First posted 15 Jun 2026cs.CL cs.AI cs.CYcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Language models (LMs) often pass behavioral bias evaluations, but it remains unclear whether they no longer represent the underlying associations that give rise to biases, or have merely learned not to express them. In this study, we show that representational biases are often detectable, even when behavioral biases are not visible. We introduce a causal framework that decomposes occupational bias into two measurement points: a model's internal representation of a user's competence, and its observable outputs. We derive steering vectors for representations of user expertise, and verify that they causally mediate model behavior in both a question-answering task and a hiring task. Applying this framework to several open-weight models, we find that demographic attributes, such as gender, race, and socioeconomic status, influence a model's representation of user expertise, even in cases where behavioral metrics detect no disparity between demographics. We show that these model representations can influence downstream behavior under intervention, suggesting failure modes that behavioral metrics alone may not detect.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 59 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

14 of 59 references.

References · 14

  1. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  2. 2023Function Vectors in Large Language Modelscited by 308
  3. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  4. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  5. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 124
  6. 2017Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American Englishcited by 71
  7. 2025Detecting High-Stakes Interactions with Activation Probescited by 53
  8. 2023Circuit Breaking: Removing Model Behaviors with Targeted Ablationcited by 27
  9. 2025Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalizationcited by 24
  10. 2025Robustly Improving LLM Fairness in Realistic Settings via Interpretabilitycited by 18
  11. 2025Investigating task-specific prompts and sparse autoencoders for activation monitoringcited by 15
  12. 2025Detecting (Un)answerability in Large Language Models with Linear Directionscited by 13
2 more
  1. 2025CRISP: Persistent Concept Unlearning via Sparse Autoencoderscited by 11
  2. 2026Features as Rewards: Scalable Supervision for Open-Ended Tasks via Interpretabilitycited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
  2. 2023Gender bias and stereotypes in Large Language Models
  3. 2025Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models
  4. 2024Interpreting Bias in Large Language Models: A Feature-Based Approach
  5. 2025Activation Steering for Bias Mitigation: An Interpretable Approach to Safer LLMs
  6. 2025Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection
  7. 2026Who Endorsed It? Measuring Authority Bias Across Expertise Levels in Language Models
  8. 2026Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation