Fair outputs, Biased Internals: Causal Potency and Asymmetry of Latent Bias in LLMs for High-Stakes Decisions
Not on the map. Why?
Abstract (arXiv)
Instruction-tuned language models exhibit behavioural fairness in high-stakes decisions while retaining biased associations in their internal representations. However, whether these suppressed representations can affect model outputs - and whether such causal potency is symmetric across demographic groups - remains unknown. We investigate the use of open-weight models for mortgage underwriting using matched applications that differ only in racially-associated names and reveal a critical disconnect: models show no output-level bias, yet retain and amplify demographic representations across model layers. Through activation steering and novel cross-layer interventions, we demonstrate that this suppressed information is decision-relevant: when reinjected at critical layers, it produces near-complete decision reversals. Critically, this latent bias is asymmetric - steering interventions affect decisions in one demographic direction, while producing minimal effects in reverse - and susceptible to adversarial prompt engineering and parameter-efficient fine-tuning. These findings demonstrate that behavioural audits focused on outputs are insufficient: fair outputs can mask exploitable internal biases. They also motivate dual-layer testing frameworks combining output evaluation with representational analysis for AI governance in high-stakes decisions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
24 of 36 references.
References · 24
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
- 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
- 2023Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Modelscited by 20
12 more
- 2025LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Languagecited by 17
- 2024Divine LLaMAs: Bias, Stereotypes, Stigmatization, and Emotion Representation of Religion in Large Language Modelscited by 12
- 2025Silenced Biases: The Dark Side LLMs Learned to Refusecited by 10
- 2025What's Not Said Still Hurts: A Description-Based Evaluation Framework for Measuring Social Bias in LLMscited by 6
- 2024Forecasting Credit Ratings: A Case Study where Traditional Methods Outperform Generative LLMscited by 5
- 2025More Women, Same Stereotypes: Unpacking the Gender Bias Paradox in Large Language Modelscited by 5
- 2025Interpretable LLMs for Credit Risk: A Systematic Review and Taxonomycited by 5
- 2025On Optimal Steering to Achieve Exact Fairnesscited by 2
- 2025Interpreting LLMs as Credit Risk Classifiers: Do Their Feature Explanations Align with Classical ML?cited by 2
- 2026Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Studycited by 2
- 2025A Large Language Model for Corporate Credit Scoringcited by 1
- 2026LendNova: Towards Automated Credit Risk Assessment with Language Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.