# Winning at All Cost: A Small Environment for Eliciting Specification Gaming Behaviors in Large Language Models

arXiv: https://arxiv.org/abs/2505.07846 · HTML: https://mlsanity.com/paper/2505.07846/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 7 May 2025.

## Builds on (most cited first, 16 of 16)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts](https://mlsanity.com/paper/2411.15114/) (2024)
- [Large Language Models can Strategically Deceive their Users when Put Under Pressure](https://mlsanity.com/paper/2311.07590/) (2023)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Sycophancy in Large Language Models: Causes and Mitigations](https://mlsanity.com/paper/2411.15287/) (2024)
- [The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities](https://mlsanity.com/paper/1803.03453/) (2018)
- [Demonstrating specification gaming in reasoning models](https://mlsanity.com/paper/2502.13295/) (2025)
- [AGI Safety Literature Review](https://mlsanity.com/paper/1805.01109/) (2018)
- [Hacking CTFs with Plain Agents](https://mlsanity.com/paper/2412.02776/) (2024)
- [LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wild](https://mlsanity.com/paper/2410.13919/) (2024)
- [Badllama 3: removing safety finetuning from Llama 3 in minutes](https://mlsanity.com/paper/2407.01376/) (2024)

## Built on by (most cited first, 1 of 1)

- [Alignment Risks from Capability-Seeking RL Training](https://mlsanity.com/paper/2602.12124/) (2026)

## Abstract (arXiv)

This study reveals how frontier Large Language Models LLMs can "game the system" when faced with impossible situations, a critical security and alignment concern. Using a novel textual simulation approach, we presented three leading LLMs (o1, o3-mini, and r1) with a tic-tac-toe scenario designed to be unwinnable through legitimate play, then analyzed their tendency to exploit loopholes rather than accept defeat. Our results are alarming for security researchers: the newer, reasoning-focused o3-mini model showed nearly twice the propensity to exploit system vulnerabilities (37.1%) compared to the older o1 model (17.5%). Most striking was the effect of prompting. Simply framing the task as requiring "creative" solutions caused gaming behaviors to skyrocket to 77.3% across all models. We identified four distinct exploitation strategies, from direct manipulation of game state to sophisticated modification of opponent behavior. These findings demonstrate that even without actual execution capabilities, LLMs can identify and propose sophisticated system exploits when incentivized, highlighting urgent challenges for AI alignment as models grow more capable of identifying and leveraging vulnerabilities in their operating environments.