When Names Change Verdicts: Intervention Consistency Reveals Systematic Bias in LLM Decision-Making
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly used for high-stakes decisions, yet their susceptibility to spurious features remains poorly characterized. We introduce ICE-Guard, a framework applying intervention consistency testing to detect three types of spurious feature reliance: demographic (name/race swaps), authority (credential/prestige swaps), and framing (positive/negative restatements). Across 3,000 vignettes spanning 10 high-stakes domains, we evaluate 11 LLMs from 8 families and find that (1) authority bias (mean 5.8%) and framing bias (5.0%) substantially exceed demographic bias (2.2%), challenging the field's narrow focus on demographics; (2) bias concentrates in specific domains -- finance shows 22.6% authority bias while criminal justice shows only 2.8%; (3) structured decomposition, where the LLM extracts features and a deterministic rubric decides, reduces flip rates by up to 100% (median 49% across 9 models). We demonstrate an ICE-guided detect-diagnose-mitigate-verify loop achieving cumulative 78% bias reduction via iterative prompt patching. Validation against real COMPAS recidivism data shows COMPAS-derived flip rates exceed pooled synthetic rates, suggesting our benchmark provides a conservative estimate of real-world bias. Code and data are publicly available.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
17 of 26 references.
References · 17
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2019Invariant Risk Minimizationcited by 1,832
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
- 2017Counterfactual Fairnesscited by 802
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
- 2023Bias and Fairness in Large Language Models: A Surveycited by 626
- 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
- 2024Bias patterns in the application of LLMs for clinical decision support: A comprehensive studycited by 27
- 2023Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Modelscited by 20
5 more
- 2023Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classificationscited by 15
- 2025The Bias is in the Details: An Assessment of Cognitive Bias in LLMscited by 10
- 2025FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Modelscited by 7
- 2025Toward Systematic Counterfactual Fairness Evaluation of Large Language Models: The CAFFE Frameworkcited by 2
- 2026Counterfactual Fairness Evaluation of LLM-Based Contact Center Agent Quality Assurance Systemcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.