Proxy reliance in large language model decisions is uncalibrated to predictive evidence
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are entering decisions in triage and lending, where task-relevant inference must be distinguished from impermissible proxy use. Current audits ask whether decisions change when demographics change. But attributes correlated with a protected group carry predictive value, so a changed decision can be discrimination or sound inference. We measure causal proxy effects in four LLMs on a clinical-ranking task with known ground truth, where the reliance the evidence warrants can be computed exactly and used as the reference. One audit signal yields three verdicts: over-reliance, warranted and under-reliance. Under neutral labels every model relies on proxies with no information. Informative proxies draw all three. Social field names push reliance down, below the reference in one model. Two findings explain this. Reliance severely undertracks the evidence, and social-label suppression is fragile, since in-context examples raise it above zero in every model. Accuracy-based evaluation detects none of this.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 49 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
37 of 49 references.
References · 37
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
- 2017Counterfactual Fairnesscited by 804
- 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
- 2022Using cognitive psychology to understand GPT-3cited by 398
- 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
25 more
- 2017Avoiding Discrimination through Causal Reasoningcited by 194
- 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 124
- 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 117
- 2022Causal Fairness Analysiscited by 43
- 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 40
- 2022Causal Conceptions of Fairness and their Consequencescited by 38
- 2025Bayesian Teaching Enables Probabilistic Reasoning in Large Language Modelscited by 37
- 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 36
- 2019Counterfactual Reasoning for Fair Clinical Risk Predictioncited by 28
- 2024Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?cited by 25
- 2023Machine Psychologycited by 22
- 2025Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMscited by 20
- 2025Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Racecited by 15
- 2025RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Datacited by 12
- 2024Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learningcited by 10
- 2026BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulationcited by 7
- 2026Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMscited by 6
- 2025On the Effectiveness and Generalization of Race Representations for Debiasing High-Stakes Decisionscited by 5
- 2026LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefscited by 5
- 2026Can LLMs Hire Fairly? Racial Bias in Resume Screeningcited by 4
- 2024Using sensitive data to de-bias AI systems: Article 10(5) of the EU AI Actcited by 3
- 2024Fair In-Context Learning via Latent Concept Variablescited by 3
- 2026EQUITRIAGE: A Fairness Audit of Gender Bias in LLM-Based Emergency Department Triagecited by 3
- 2026Uncovering Latent Bias in LLM-Based Emergency Department Triage Through Proxy Variablescited by 2
- 2026PopResume: Causal Fairness Evaluation of LLM/VLM Resume Screeners with Population-Representative Datasetcited by 1
Built on by
Not cited yet.