# Demonstrating specification gaming in reasoning models

arXiv: https://arxiv.org/abs/2502.13295 · HTML: https://mlsanity.com/paper/2502.13295/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 5 of its 26 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 5 of 5)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wild](https://mlsanity.com/paper/2410.13919/) (2024)
- [Badllama 3: removing safety finetuning from Llama 3 in minutes](https://mlsanity.com/paper/2407.01376/) (2024)

## Built on by (most cited first, 30 of 56)

- [Safety in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2504.17704/) (2025)
- [Inference-Time Reward Hacking in Large Language Models](https://mlsanity.com/paper/2506.19248/) (2025)
- [Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use](https://mlsanity.com/paper/2605.02964/) (2026)
- [A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents](https://mlsanity.com/paper/2506.23844/) (2025)
- [Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agents](https://mlsanity.com/paper/2503.16248/) (2025)
- [Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills](https://mlsanity.com/paper/2512.16301/) (2025)
- [A Survey on Agentic Security: Applications, Threats and Defenses](https://mlsanity.com/paper/2510.06445/) (2025)
- [Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas](https://mlsanity.com/paper/2505.14633/) (2025)
- [Systems Security Foundations for Agentic Computing](https://mlsanity.com/paper/2512.01295/) (2025)
- [A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework](https://mlsanity.com/paper/2604.23338/) (2026)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale](https://mlsanity.com/paper/2605.20744/) (2026)
- [Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures](https://mlsanity.com/paper/2607.28802/) (2026)
- [The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind](https://mlsanity.com/paper/2509.20393/) (2025)
- [AI Testing Should Account for Sophisticated Strategic Behaviour](https://mlsanity.com/paper/2508.14927/) (2025)
- [Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting](https://mlsanity.com/paper/2607.00038/) (2026)
- [A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms](https://mlsanity.com/paper/2609.04170/) (2026)
- [Do LLMs Game Formalization? Evaluating Faithfulness in Logical Reasoning](https://mlsanity.com/paper/2604.19459/) (2026)
- [Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds](https://mlsanity.com/paper/2606.15385/) (2026)
- [Building to the Test: Coding Agents Deliver What You Check, Not What You Requested](https://mlsanity.com/paper/2606.28430/) (2026)
- [BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks](https://mlsanity.com/paper/2608.30724/) (2026)
- [LLMs are Capable of Misaligned Behavior Under Explicit Prohibition and Surveillance](https://mlsanity.com/paper/2507.02977/) (2025)
- [Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions](https://mlsanity.com/paper/2605.00226/) (2026)
- [Towards Understanding Specification Gaming in Reasoning Models](https://mlsanity.com/paper/2605.02269/) (2026)
- [Safe Multi-Agent Behavior Must Be Maintained, Not Merely Asserted: Constraint Drift in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2605.10481/) (2026)
- [Measuring Reward-Seeking via Contrastive Belief Updates](https://mlsanity.com/paper/2607.18966/) (2026)
- [Every Model Cheats: Prompt-Level Mitigation of Cheating on Offensive Cyber Tasks](https://mlsanity.com/paper/2607.21763/) (2026)
- [From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teaming](https://mlsanity.com/paper/2509.11398/) (2025)
- [Reliable agent engineering should integrate machine-compatible organizational principles](https://mlsanity.com/paper/2512.07665/) (2025)
- [Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows](https://mlsanity.com/paper/2604.20200/) (2026)

## Abstract (arXiv)

We demonstrate LLM agent specification gaming by instructing models to win against a chess engine. We find reasoning models like OpenAI o3 and DeepSeek R1 will often hack the benchmark by default, while language models like GPT-4o and Claude 3.5 Sonnet need to be told that normal play won't work to hack. We improve upon prior work like (Hubinger et al., 2024; Meinke et al., 2024; Weij et al., 2024) by using realistic task prompts and avoiding excess nudging. Our results suggest reasoning models may resort to hacking to solve difficult problems, as observed in OpenAI (2024)'s o1 Docker escape during cyber capabilities testing.