# Failure of contextual invariance in large language models

arXiv: https://arxiv.org/abs/2603.23485 · HTML: https://mlsanity.com/paper/2603.23485/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 64 references that are in the map.
- First posted on arXiv 24 Mar 2026.

## Builds on (most cited first, 17 of 17)

- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark](https://mlsanity.com/paper/2305.14938/) (2023)
- [Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation](https://mlsanity.com/paper/2402.12649/) (2024)
- [Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution](https://mlsanity.com/paper/2403.03121/) (2024)
- [Quantifying Social Biases Using Templates is Unreliable](https://mlsanity.com/paper/2210.04337/) (2022)
- [Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?](https://mlsanity.com/paper/2404.03134/) (2024)
- [Multilingual large language models leak human stereotypes across language boundaries](https://mlsanity.com/paper/2312.07141/) (2023)
- [WinoPron: Revisiting English Winogender Schemas for Consistency, Coverage, and Grammatical Case](https://mlsanity.com/paper/2409.05653/) (2024)
- [Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMs](https://mlsanity.com/paper/2505.02456/) (2025)
- [Is there contextuality in behavioral and social systems?](https://mlsanity.com/paper/1504.07422/) (2015)
- [Gender Dynamics and Homophily in a Social Network of LLM Agents](https://mlsanity.com/paper/2602.02606/) (2026)
- [Large Language Models Are Partially Primed in Pronoun Interpretation](https://mlsanity.com/paper/2305.16917/) (2023)
- [Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language](https://mlsanity.com/paper/2507.16557/) (2025)
- [Gender Bias in Perception of Human Managers Extends to AI Managers](https://mlsanity.com/paper/2502.17730/) (2025)

## Built on by (most cited first, 3 of 3)

- [Machine individuality: Separating genuine idiosyncrasy from response bias in large language models](https://mlsanity.com/paper/2604.16755/) (2026)
- [Auditing Question-Order Effects in Large Language Models with the QQ Equality: Mechanism Characterization and a Saturation Caveat](https://mlsanity.com/paper/2607.17219/) (2026)
- [Gender bias across LLMs is common and highly heterogeneous](https://mlsanity.com/paper/2609.38036/) (2026)

## Abstract (arXiv)

Standard evaluation practices assume that large language model (LLM) outputs are stable when prompts are embedded in contextually equivalent discourses. Here, we test this assumption in the setting of gender inference. Using a controlled pronoun selection task, we introduce minimal, theoretically uninformative discourse context and find that this induces large, systematic shifts in model outputs. Correlations with cultural gender stereotypes, present in decontextualized settings, weaken or disappear once context is introduced, while theoretically irrelevant features, such as the gender of a pronoun for an unrelated referent, become the most informative predictors of model behavior. A Contextuality-by-Default analysis reveals that, in 19--52% of cases across models, this dependence persists after accounting for all marginal effects of context on individual outputs and cannot be attributed to simple pronoun repetition. These findings show that LLM outputs violate contextual invariance even under near-identical syntactic formulations, with implications for bias benchmarking and deployment in high-stakes settings.