mlsanityData to 14 Oct 2026

When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Making

arXiv 2610.19948 ↗First posted 14 Oct 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Frontier language models excel at complex tasks yet may prioritize protection over larger physical consequences in simple, urgent decisions. GPT-5.6 Sol, for example, moves a girl playing in shallow water before retrieving a submerged camera. We call this Protection Mode and study it with ProtectionBench: 285 prompts, 34 physical families, and nine models. At lowest effort, at least one of four responses prioritizes the subject on 34.74% of Sol prompts and over 83% of MiniMax M3 and Qwen3.5-27B prompts; none of four independent human raters does so. Controls show that choices usually track physical risk. We elicit both risks before requesting an action. Some models assess risk incorrectly; others state the correct ordering but still choose the subject. Qwen3.5-27B with thinking shows 100% correct risk ordering but 57.81% subject choice among eligible conversations on eight selected scenes. Attention audits find no stable reduction in attention to relevant facts; removing the stated risk direction reduces object-first choice in dots3-note-prev but not detectably in Qwen3.5-27B. Adding a general "people or animals before property" rule increases subject choice by 19-38 percentage points, whereas factual reminders favor the object by 0.7-5.2 points. Removing naturally generated versions of that rule lowers subject choice by 10-15 points. We also examine whether post-training introduces this pattern. Base models already show Protection Mode; post-training changes justifications more than choices.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 51 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

35 of 51 references.

References · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  9. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  10. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  11. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  12. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
23 more
  1. 2022Red Teaming Language Models with Language Modelscited by 1,026
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  3. 2020Aligning AI With Shared Human Valuescited by 723
  4. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  5. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  6. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  8. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  9. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  10. 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
  11. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  12. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  13. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
  14. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  15. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  16. 2025Monitoring Monitorabilitycited by 58
  17. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 43
  18. 2025Safety Pretraining: Toward the Next Generation of Safe AIcited by 43
  19. 2026OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Taskscited by 40
  20. 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbositycited by 17
  21. 2025ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMscited by 6
  22. 2026SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agentscited by 5
  23. 2026The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMscited by 1

Built on by

Not cited yet.