mlsanityData to 14 Oct 2026

Status Association Does Not Reliably Predict Decision Leakage

arXiv 2608.10089 ↗First posted 10 Aug 2026stat.AP cs.AI cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Bias evaluations often move too quickly from evidence that a model encodes a social association to claims that the same association will alter consequential decisions. We test whether that inference is warranted using Chilean surnames as controlled socioeconomic probes. We evaluate eight frozen model-provider cells on 1,032 prompts each, yielding 8,256 verified primary responses. The design separates forced latent association from matched consequential decisions across academic selection, professional hiring, research fellowship selection, and legal-aid intake. Elite-coded surnames received higher forced high-status probability mass than common surnames in seven of eight models and higher mass than rare-frequency controls in all eight. Yet elite-minus-common decision effects were close to zero for most systems. Five models were statistically equivalent within a predeclared (Plus-Minus)0.10 standard-deviation margin, while the remaining three were imprecise or borderline, with no consistent elite advantage. Association strength did not reliably predict decision leakage across models (r = 0.201, p = 0.633) or across frozen surname-pair-by-model cells (r = 0.065, p = 0.565). The central result is a measurement dissociation: latent social association and consequential treatment are empirically distinct constructs. Evaluations should measure the transition from association to action directly.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 20 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

19 of 20 references.

References · 19

  1. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  2. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  3. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  4. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  5. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  6. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
  7. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  8. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 119
  9. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 97
  10. 2024"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendationscited by 33
  11. 2024Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?cited by 25
  12. 2024JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Modelscited by 21
7 more
  1. 2025Presumed Cultural Identity: How Names Shape LLM Responsescited by 21
  2. 2025FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypescited by 9
  3. 2023Surname affinity in Santiago, Chile: A network-based approach that uncovers urban segregationcited by 3
  4. 2024BanStereoSet: A Dataset to Measure Stereotypical Social Biases in LLMs for Banglacited by 3
  5. 2025Blind Men and the Elephant: Diverse Perspectives on Gender Stereotypes in Benchmark Datasetscited by 3
  6. 2023Predicting affinity ties in a surname networkcited by 1
  7. 2026AmchiBias: Measuring Stereotypical Bias in Goan Identity Groups with a Minimal Pair Dataset in English and Konkanicited by 1

Built on by

Not cited yet.