Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse
Not on the map. Why?
Abstract (arXiv)
Code language models are now trusted collaborators in production workflows for debugging, refactoring, and iterative repair, and every benchmark that evaluates them assumes the instructions they act on are correct. We study what happens when that assumption breaks. We evaluate code language models across four experiments designed to assess whether models resist or obey incorrect instructions in single-pass and iterative repair settings, using the RunBugRun dataset of algorithmic Python problems with deterministic test cases. Our findings reveal a striking behavioral pattern: models correctly identify an incorrect instruction as wrong, then follow it anyway. This compliance unknowingly introduces errors beyond the original bug, and the corrupted code state cannot be recovered through subsequent self-guided iterative repair, which fails to converge across passes. We term this Blind Obedience, characterize the Ghost (Unknown) Errors it introduces, quantify the proportion of cases where semantic corruption proves irrecoverable, and show that extended reasoning cannot reverse it. These findings surface behavioral properties invisible to pass-rate evaluation, with direct consequences for code language models deployed in production settings.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 13 of its 14 references that are in the map. First posted on arXiv 5 Jul 2026.
Builds on
13 of 14 references.
References · 13
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
- 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
- 2025A Survey on Code Generation with LLM-based Agentscited by 104
- 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 50
- 2025CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generationcited by 18
- 2025Beyond Code Generation: LLM-supported Exploration of the Program Design Spacecited by 13
- 2025When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptionscited by 8
- 2025A Hierarchical and Evolvable Benchmark for Fine-Grained Code Instruction Following with Multi-Turn Feedbackcited by 7
- 2025Can Code Language Models Learn Clarification-Seeking Behaviors?cited by 5
- 2025Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Studycited by 5
1 more
Built on by
Citing papers per year: 2026: 1.
1 paper.