When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Making
Not on the map. Why?
Abstract (arXiv)
Frontier language models excel at complex tasks yet may prioritize protection over larger physical consequences in simple, urgent decisions. GPT-5.6 Sol, for example, moves a girl playing in shallow water before retrieving a submerged camera. We call this Protection Mode and study it with ProtectionBench: 285 prompts, 34 physical families, and nine models. At lowest effort, at least one of four responses prioritizes the subject on 34.74% of Sol prompts and over 83% of MiniMax M3 and Qwen3.5-27B prompts; none of four independent human raters does so. Controls show that choices usually track physical risk. We elicit both risks before requesting an action. Some models assess risk incorrectly; others state the correct ordering but still choose the subject. Qwen3.5-27B with thinking shows 100% correct risk ordering but 57.81% subject choice among eligible conversations on eight selected scenes. Attention audits find no stable reduction in attention to relevant facts; removing the stated risk direction reduces object-first choice in dots3-note-prev but not detectably in Qwen3.5-27B. Adding a general "people or animals before property" rule increases subject choice by 19-38 percentage points, whereas factual reminders favor the object by 0.7-5.2 points. Removing naturally generated versions of that rule lowers subject choice by 10-15 points. We also examine whether post-training introduces this pattern. Base models already show Protection Mode; post-training changes justifications more than choices.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 51 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
35 of 51 references.
References · 35
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2023Let's Verify Step by Stepcited by 3,789
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
23 more
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
- 2020Aligning AI With Shared Human Valuescited by 723
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2025Reasoning Models Don't Always Say What They Thinkcited by 415
- 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
- 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
- 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
- 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
- 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
- 2025Monitoring Monitorabilitycited by 58
- 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 43
- 2025Safety Pretraining: Toward the Next Generation of Safe AIcited by 43
- 2026OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Taskscited by 40
- 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbositycited by 17
- 2025ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMscited by 6
- 2026SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agentscited by 5
- 2026The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMscited by 1
Built on by
Not cited yet.