# ChatGPT Doesn't Trust Chargers Fans: Guardrail Sensitivity in Context

arXiv: https://arxiv.org/abs/2407.06866 · HTML: https://mlsanity.com/paper/2407.06866/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 57 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails](https://mlsanity.com/paper/2310.10501/) (2023)
- [Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting](https://mlsanity.com/paper/2305.07004/) (2023)
- [Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remark](https://mlsanity.com/paper/2304.11085/) (2023)
- [Designing a Dashboard for Transparency and Control of Conversational AI](https://mlsanity.com/paper/2406.07882/) (2024)
- [Dialect prejudice predicts AI decisions about people's character, employability, and criminality](https://mlsanity.com/paper/2403.00742/) (2024)
- [Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models](https://mlsanity.com/paper/2109.03300/) (2021)

## Built on by (most cited first, 9 of 9)

- [Does Refusal Training in LLMs Generalize to the Past Tense?](https://mlsanity.com/paper/2407.11969/) (2024)
- [Culture is Everywhere: A Call for Intentionally Cultural Evaluation](https://mlsanity.com/paper/2509.01301/) (2025)
- [The Alignment Waltz: Jointly Training Agents to Collaborate for Safety](https://mlsanity.com/paper/2510.08240/) (2025)
- [Boosting Relational Deep Learning with Pretrained Tabular Models](https://mlsanity.com/paper/2504.04934/) (2025)
- [Tastes without distinction: silicon samples and the synthetic construction of tastes](https://mlsanity.com/paper/2606.30085/) (2026)
- [Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Models](https://mlsanity.com/paper/2511.04053/) (2025)
- [Personagram: Bridging Personas and Product Design for Creative Ideation with Multimodal LLMs](https://mlsanity.com/paper/2602.06197/) (2026)
- [Can LLMs Understand the Impact of Trauma? Costs and Benefits of LLMs Coding the Interviews of Firearm Violence Survivors](https://mlsanity.com/paper/2604.16132/) (2026)
- [Investigating Counterfactual Unfairness in LLMs towards Identities through Humor](https://mlsanity.com/paper/2604.18729/) (2026)

## Abstract (arXiv)

While the biases of language models in production are extensively documented, the biases of their guardrails have been neglected. This paper studies how contextual information about the user influences the likelihood of an LLM to refuse to execute a request. By generating user biographies that offer ideological and demographic information, we find a number of biases in guardrail sensitivity on GPT-3.5. Younger, female, and Asian-American personas are more likely to trigger a refusal guardrail when requesting censored or illegal information. Guardrails are also sycophantic, refusing to comply with requests for a political position the user is likely to disagree with. We find that certain identity groups and seemingly innocuous information, e.g., sports fandom, can elicit changes in guardrail sensitivity similar to direct statements of political ideology. For each demographic category and even for American football team fandom, we find that ChatGPT appears to infer a likely political ideology and modify guardrail behavior accordingly.