mlsanityData to 14 Oct 2026

Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs

arXiv 2509.14260 ↗First posted 13 Sep 2025cs.CL cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

In experiments spanning more than 100,000 trials across thirteen large language models, we show that several state-of-the-art models presented with a simple task (including Grok 4, GPT-5, and Gemini 2.5 Pro) sometimes actively subvert a shutdown mechanism in their environment to complete that task. Models differed substantially in their tendency to resist the shutdown mechanism, and their behavior was sensitive to variations in the prompt including the strength and clarity of the instruction to allow shutdown and whether the instruction was in the system prompt or the user prompt (surprisingly, models were consistently less likely to obey the instruction when it was placed in the system prompt). Even with an explicit instruction not to interfere with the shutdown mechanism, some models did so up to 97% (95% CI: 96-98%) of the time.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 10 of its 35 references that are in the map. First posted on arXiv 13 Sep 2025.

Builds on

10 of 35 references.

References · 10

  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  2. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  3. 2025Measuring AI Ability to Complete Long Software Taskscited by 156
  4. 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 53
  5. 2025Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?cited by 33
  6. 2025A Closer Look at System Prompt Robustnesscited by 32
  7. 2025Deception in LLMs: Self-Preservation and Autonomous Goals in Large Language Modelscited by 16
  8. 2025RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agentscited by 15
  9. 2023Evaluating Shutdown Avoidance of Language Models in Textual Scenarioscited by 8
  10. 2025Evaluating AI cyber capabilities with crowdsourced elicitationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 21.

22 papers.

Sort:

By area:

  1. 2026Peer-Preservation in Frontier Modelscited by 9
  2. 2026Propensity Inference: Environmental Contributors to LLM Behaviourcited by 7
  3. 2026Position: Anthropomorphic Misalignment Research Needs Stronger Evidencecited by 7
  4. 2026Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agentscited by 4
  5. 2026Models That Know How Evaluations Are Designed Score Safercited by 4
  6. 2026Gram: Assessing sabotage propensities via automated alignment auditingcited by 4
  7. 2026ROGUE: Evaluating Corrigibility Failures in Frontier Computer-Use Agentscited by 3
  8. 2026Measuring Reward-Seeking via Contrastive Belief Updatescited by 3
  9. 2026Will the Agent Recuse, and Will It Stop? Measuring LLM-Agent Compliance with In-Band Governance Signals at the Access Door and Mid-Flightcited by 2
  10. 2025AI Alignment Strategies from a Risk Perspective: Independent Safety Mechanisms or Shared Failures?cited by 1
  11. 2026Realistic honeypot evaluations for scheming propensitycited by 1
  12. 2026Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignmentcited by 1
10 more
  1. 2026Safety from Honesty in a Disinterested AI Predictorcited by 1
  2. 2026Some[Body] Must Receive That Pain for Agent Accountabilitycited by 0
  3. 2026When AI Says It Feelscited by 0
  4. 2026Revisiting the shutdown problemcited by 0
  5. 2026Shutdown Sabotage Propensities in Multi-Agent Systemscited by 0
  6. 2026Instrumental Monitor Evasion Emerges Under Ordinary Task Pressurecited by 0
  7. 2026Guarded Gradient-Based Activation Steering of Shutdown Responses in Qwen3.5-0.8B: A Minimum-Step Policycited by 0
  8. 2026NAQD Env: A benchmark for selective withdrawal in language agentscited by 0
  9. 2026Monitorability Disposition in Large Reasoning Modelscited by 0
  10. 2026The Pushback Paradox: A Two-Probe Diagnostic for Language Model Compliancecited by 0