# Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs

arXiv: https://arxiv.org/abs/2509.14260 · HTML: https://mlsanity.com/paper/2509.14260/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 13 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [Measuring AI Ability to Complete Long Software Tasks](https://mlsanity.com/paper/2503.14499/) (2025)
- [IHEval: Evaluating Language Models on Following the Instruction Hierarchy](https://mlsanity.com/paper/2502.08745/) (2025)
- [Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?](https://mlsanity.com/paper/2502.12206/) (2025)
- [A Closer Look at System Prompt Robustness](https://mlsanity.com/paper/2502.12197/) (2025)
- [Deception in LLMs: Self-Preservation and Autonomous Goals in Large Language Models](https://mlsanity.com/paper/2501.16513/) (2025)
- [RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents](https://mlsanity.com/paper/2504.18565/) (2025)
- [Evaluating Shutdown Avoidance of Language Models in Textual Scenarios](https://mlsanity.com/paper/2307.00787/) (2023)
- [Evaluating AI cyber capabilities with crowdsourced elicitation](https://mlsanity.com/paper/2505.19915/) (2025)

## Built on by (most cited first, 22 of 22)

- [Peer-Preservation in Frontier Models](https://mlsanity.com/paper/2604.19784/) (2026)
- [Propensity Inference: Environmental Contributors to LLM Behaviour](https://mlsanity.com/paper/2604.21098/) (2026)
- [Position: Anthropomorphic Misalignment Research Needs Stronger Evidence](https://mlsanity.com/paper/2606.07612/) (2026)
- [Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents](https://mlsanity.com/paper/2605.16282/) (2026)
- [Models That Know How Evaluations Are Designed Score Safer](https://mlsanity.com/paper/2605.28591/) (2026)
- [Gram: Assessing sabotage propensities via automated alignment auditing](https://mlsanity.com/paper/2605.30322/) (2026)
- [ROGUE: Evaluating Corrigibility Failures in Frontier Computer-Use Agents](https://mlsanity.com/paper/2606.00341/) (2026)
- [Measuring Reward-Seeking via Contrastive Belief Updates](https://mlsanity.com/paper/2607.18966/) (2026)
- [Will the Agent Recuse, and Will It Stop? Measuring LLM-Agent Compliance with In-Band Governance Signals at the Access Door and Mid-Flight](https://mlsanity.com/paper/2606.06460/) (2026)
- [AI Alignment Strategies from a Risk Perspective: Independent Safety Mechanisms or Shared Failures?](https://mlsanity.com/paper/2510.11235/) (2025)
- [Realistic honeypot evaluations for scheming propensity](https://mlsanity.com/paper/2605.29729/) (2026)
- [Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment](https://mlsanity.com/paper/2606.26071/) (2026)
- [Safety from Honesty in a Disinterested AI Predictor](https://mlsanity.com/paper/2606.29657/) (2026)
- [Some[Body] Must Receive That Pain for Agent Accountability](https://mlsanity.com/paper/2605.16872/) (2026)
- [When AI Says It Feels](https://mlsanity.com/paper/2606.05734/) (2026)
- [Revisiting the shutdown problem](https://mlsanity.com/paper/2606.08296/) (2026)
- [Shutdown Sabotage Propensities in Multi-Agent Systems](https://mlsanity.com/paper/2609.28274/) (2026)
- [Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure](https://mlsanity.com/paper/2609.30217/) (2026)
- [Guarded Gradient-Based Activation Steering of Shutdown Responses in Qwen3.5-0.8B: A Minimum-Step Policy](https://mlsanity.com/paper/2609.30326/) (2026)
- [NAQD Env: A benchmark for selective withdrawal in language agents](https://mlsanity.com/paper/2609.38460/) (2026)
- [Monitorability Disposition in Large Reasoning Models](https://mlsanity.com/paper/2610.04914/) (2026)
- [The Pushback Paradox: A Two-Probe Diagnostic for Language Model Compliance](https://mlsanity.com/paper/2610.06673/) (2026)

## Abstract (arXiv)

In experiments spanning more than 100,000 trials across thirteen large language models, we show that several state-of-the-art models presented with a simple task (including Grok 4, GPT-5, and Gemini 2.5 Pro) sometimes actively subvert a shutdown mechanism in their environment to complete that task. Models differed substantially in their tendency to resist the shutdown mechanism, and their behavior was sensitive to variations in the prompt including the strength and clarity of the instruction to allow shutdown and whether the instruction was in the system prompt or the user prompt (surprisingly, models were consistently less likely to obey the instruction when it was placed in the system prompt). Even with an explicit instruction not to interfere with the shutdown mechanism, some models did so up to 97% (95% CI: 96-98%) of the time.