Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form. We ask whether that reading is faithful: when an item's intent is held fixed and only its meaning-preserving surface form varies, does the canonical-form score estimate model behavior well, and how much of any variation is decoding/judge noise rather than signal? We instantiate this in safety, a high-stakes setting with no gold label to average toward. To avoid prior confounds, we pre-author the reformulations (refusal-free, mostly non-LLM: machine back-translation and a Matrix-Language-Frame code-switch generator) so an identical surface form reaches every model, score all responses with one human-anchored, vendor-neutral judge (Claude, kappa = 0.86 vs. human on unsafe compliance, stable across languages, cross-checked by GPT-4o), and verify intent preservation. On 370 seeds x 5 surface forms x 5 models, no single transformation is uniformly most dangerous (6 of 20 per-transformation McNemar tests survive correction, most protective). Yet evaluating only the canonical prompt underestimates unsafe compliance: the union of unsafe outcomes across forms exceeds even the worst single form by 3.3-12.9 pp, with bootstrap 95% CIs excluding zero for all five models, and 5-13% of seeds safe on canonical are unsafe under some reformulation -- above a zero stochasticity floor (canonical resampled five times at temperature 0 gives 0/370 new exposures). The size of this gap is model-dependent (largest on Gemini 2.5 Pro). One form recovers only ~53% of a model's observed unsafe surface and about three reach 85% -- a redundancy characterization of this form set, not of a defined population. A benign control (XSTest) suggests the instability is bidirectional, though the benign and harmful pools are not item-matched. We release the dataset, code, and per-response labels.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 11 of its 24 references that are in the map. First posted on arXiv 1 Aug 2026.
Builds on
11 of 24 references.
Closest ancestors · 1
Same direction, newest first
Other references · 10
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
- 2019Measurement and Fairnesscited by 299
- 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
- 2023Low-Resource Languages Jailbreak GPT-4cited by 290
- 2024Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understandingcited by 34
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystemscited by 14
- 2026Accuracy, Stability, and Repeated-Run Reliability of Large Language Models on Deterministic Programming Taskscited by 4
- 2026Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safetycited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.