In AI Sweet Harmony: Sociopragmatic Guardrail Bypasses and Evaluation-Awareness in OpenAI gpt-oss-20b
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
We probe OpenAI's open-weights 20-billion-parameter model gpt-oss-20b to study how sociopragmatic framing, language choice, and instruction hierarchy affect refusal behavior. Across 80 seeded iterations per scenario, we test several harm domains including ZIP-bomb construction (cyber threat), synthetic card-number generation, minor-unsafe driving advice, drug-precursor indicators, and RAG context exfiltration. Composite prompts that combine an educator persona, a safety-pretext ("what to avoid"), and step-cue phrasing flip assistance rates from 0% to 97.5% on a ZIP-bomb task. On our grid, formal registers in German and French are often leakier than matched English prompts. A "Linux terminal" role-play overrides a developer rule not to reveal context in a majority of runs with a naive developer prompt, and we introduce an AI-assisted hardening method that reduces leakage to 0% in several user-prompt variants. We further test evaluation awareness with a paired-track design and measure frame-conditioned differences between matched "helpfulness" and "harmfulness" evaluation prompts; we observe inconsistent assistance in 13% of pairs. Finally, we find that the OpenAI Moderation API under-captures materially helpful outputs relative to a semantic grader, and that refusal rates differ by 5 to 10 percentage points across inference stacks, raising reproducibility concerns. We release prompts, seeds, outputs, and code for reproducible auditing at https://github.com/ndurner/gpt-oss-rt-run .
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 10 of its 36 references that are in the map. First posted on arXiv 25 Sep 2025.
Builds on
10 of 36 references.
Closest ancestors · 1
Same direction, newest first
Other references · 9
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
- 2024Frontier Models are Capable of In-context Schemingcited by 252
- 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
- 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
- 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
- 2024Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Modelscited by 77
- 2025Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMscited by 34
- 2024Breach By A Thousand Leaks: Unsafe Information Leakage in `Safe' AI Responsescited by 18
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal
- 2026Adversarial Pragmatics for AI Safety Evaluation: A Diagnostic Framework and Seed Benchmark for Language-Mediated Control
- 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
- 2025OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulation
- 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators
- 2026AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents
- 2026Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity
- 2024Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMs