# Reasoning Model is Stubborn: Diagnosing Instruction Overriding in Reasoning Models

arXiv: https://arxiv.org/abs/2505.17225 · HTML: https://mlsanity.com/paper/2505.17225/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Process Reinforcement through Implicit Rewards](https://mlsanity.com/paper/2502.01456/) (2025)
- [Llemma: An Open Language Model For Mathematics](https://mlsanity.com/paper/2310.10631/) (2023)
- [MathPrompter: Mathematical Reasoning using Large Language Models](https://mlsanity.com/paper/2303.05398/) (2023)
- [ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning](https://mlsanity.com/paper/2002.04326/) (2020)
- [CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text](https://mlsanity.com/paper/1908.06177/) (2019)
- [RM-R1: Reward Modeling as Reasoning](https://mlsanity.com/paper/2505.02387/) (2025)
- [Process Reward Models That Think](https://mlsanity.com/paper/2504.16828/) (2025)
- [o1-Coder: an o1 Replication for Coding](https://mlsanity.com/paper/2412.00154/) (2024)
- [LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning](https://mlsanity.com/paper/2007.08124/) (2020)
- [Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoning](https://mlsanity.com/paper/2502.04381/) (2025)
- [Easy Problems That LLMs Get Wrong](https://mlsanity.com/paper/2405.19616/) (2024)
- [Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slow](https://mlsanity.com/paper/2408.08651/) (2024)
- [Do Chains-of-Thoughts of Large Language Models Suffer from Hallucinations, Cognitive Biases, or Phobias in Bayesian Reasoning?](https://mlsanity.com/paper/2503.15268/) (2025)

## Built on by (most cited first, 6 of 6)

- [The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?](https://mlsanity.com/paper/2601.23045/) (2026)
- [Context Learning for Multi-Agent Discussion](https://mlsanity.com/paper/2602.02350/) (2026)
- [On the Robustness of Answer Formats in Medical Reasoning Models](https://mlsanity.com/paper/2509.20866/) (2025)
- [Reasoning Model Is Superior LLM-Judge, Yet Suffers from Biases](https://mlsanity.com/paper/2601.03630/) (2026)
- [Input-Time Scaling: Adding Noise and Irrelevance into Less-Is-More Drastically Improves Reasoning Performance and Efficiency](https://mlsanity.com/paper/2508.13654/) (2025)
- [Mitigating Cognitive Inertia in Large Reasoning Models via Latent Spike Steering](https://mlsanity.com/paper/2601.22484/) (2026)

## Abstract (arXiv)

Large language models have demonstrated remarkable proficiency in long and complex reasoning tasks. However, they frequently exhibit a problematic reliance on familiar reasoning patterns, a phenomenon we term textitreasoning rigidity. Despite explicit instructions from users, these models often override clearly stated conditions and default to habitual reasoning trajectories, leading to incorrect conclusions. This behavior presents significant challenges, particularly in domains such as mathematics and logic puzzle, where precise adherence to specified constraints is critical. To systematically investigate reasoning rigidity, a behavior largely unexplored in prior work, we introduce a expert-curated diagnostic set, dataset. Our dataset includes specially modified variants of existing mathematical benchmarks, namely AIME and MATH500, as well as well-known puzzles deliberately redesigned to require deviation from familiar reasoning strategies. Using this dataset, we identify recurring contamination patterns that occur when models default to ingrained reasoning. Specifically, we categorize this contamination into three distinctive modes: (i) Interpretation Overload, (ii) Input Distrust, and (iii) Partial Instruction Attention, each causing models to ignore or distort provided instructions. We publicly release our diagnostic set to facilitate future research on mitigating reasoning rigidity in language models.