mlsanityData to 14 Oct 2026

Demonstrating specification gaming in reasoning models

arXiv 2502.13295 ↗First posted 18 Feb 2025cs.AIcited by 56

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

We demonstrate LLM agent specification gaming by instructing models to win against a chess engine. We find reasoning models like OpenAI o3 and DeepSeek R1 will often hack the benchmark by default, while language models like GPT-4o and Claude 3.5 Sonnet need to be told that normal play won't work to hack. We improve upon prior work like (Hubinger et al., 2024; Meinke et al., 2024; Weij et al., 2024) by using realistic task prompts and avoiding excess nudging. Our results suggest reasoning models may resort to hacking to solve difficult problems, as observed in OpenAI (2024)'s o1 Docker escape during cyber capabilities testing.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 5 of its 26 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

5 of 26 references.

References · 5

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  3. 2023How is ChatGPT's behavior changing over time?cited by 270
  4. 2024LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wildcited by 12
  5. 2024Badllama 3: removing safety finetuning from Llama 3 in minutescited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 36.

56 papers.

Sort:

By area:

  1. 2025Safety in Large Reasoning Models: A Surveycited by 64
  2. 2025Inference-Time Reward Hacking in Large Language Modelscited by 34
  3. 2026Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Usecited by 31
  4. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 29
  5. 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 24
  6. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
  7. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 20
  8. 2025Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmascited by 18
  9. 2025Systems Security Foundations for Agentic Computingcited by 13
  10. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 11
  11. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 10
  12. 2026Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scalecited by 10
38 more
  1. 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failurescited by 10
  2. 2025The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blindcited by 8
  3. 2025AI Testing Should Account for Sophisticated Strategic Behaviourcited by 6
  4. 2026Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Promptingcited by 5
  5. 2026A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarmscited by 5
  6. 2026Do LLMs Game Formalization? Evaluating Faithfulness in Logical Reasoningcited by 4
  7. 2026Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworldscited by 4
  8. 2026Building to the Test: Coding Agents Deliver What You Check, Not What You Requestedcited by 4
  9. 2026BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Taskscited by 4
  10. 2025LLMs are Capable of Misaligned Behavior Under Explicit Prohibition and Surveillancecited by 3
  11. 2026Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actionscited by 3
  12. 2026Towards Understanding Specification Gaming in Reasoning Modelscited by 3
  13. 2026Safe Multi-Agent Behavior Must Be Maintained, Not Merely Asserted: Constraint Drift in LLM-Based Multi-Agent Systemscited by 3
  14. 2026Measuring Reward-Seeking via Contrastive Belief Updatescited by 3
  15. 2026Every Model Cheats: Prompt-Level Mitigation of Cheating on Offensive Cyber Taskscited by 3
  16. 2025From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teamingcited by 2
  17. 2025Reliable agent engineering should integrate machine-compatible organizational principlescited by 2
  18. 2026Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflowscited by 2
  19. 2026Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluationscited by 2
  20. 2025Winning at All Cost: A Small Environment for Eliciting Specification Gaming Behaviors in Large Language Modelscited by 1
  21. 2025RRTL: Red Teaming Reasoning Large Language Models in Tool Learningcited by 1
  22. 2025Do LLMs Know They Are Being Tested? Evaluation Awareness and Incentive-Sensitive Failures in GPT-OSS-20Bcited by 1
  23. 2026Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignmentcited by 1
  24. 2026SysAdmin: Measuring Instrumental Power-Seeking in Frontier AIcited by 1
  25. 2026Reward Hacking and Agent Containment Failure: A Monte Carlo Study Based on the 2026 Hugging Face Incidentcited by 1
  26. 2025Confidence-Guided Human-AI Collaboration: Reinforcement Learning with Distributional Proxy Value Propagation for Autonomous Drivingcited by 0
  27. 2025Take Goodhart Seriously: Principled Limit on General-Purpose AI Optimizationcited by 0
  28. 2025Does Self-Evaluation Enable Wireheading in Language Models?cited by 0
  29. 2026The Missing Half: Unveiling Training-time Implicit Safety Risks Beyond Deploymentcited by 0
  30. 2026Multi-Stream Perturbation Attack: Breaking Safety Alignment of Thinking LLMs Through Concurrent Task Interferencecited by 0
  31. 2026Reframing AI Loss of Control: What Control Is, How to Have It, How to Lose Itcited by 0
  32. 2026Specifying AI-SDLC Processes: A Protocol Language for Human-Agent Boundariescited by 0
  33. 2026Defeat Devices in AI Systemscited by 0
  34. 2026EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failurescited by 0
  35. 2026Semantic Alignment of AI Models: Concept Collapse, Checkpoint Dynamics, and Cross-Lingual Transfercited by 0
  36. 2026Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Taskscited by 0
  37. 2026Reducing Catastrophic Risk from AI with Systematic Monitoring and Evaluation of Rogue AI Progressioncited by 0
  38. 2026Inducing Emergent Misalignment from Reward Hacks with Iterative DPOcited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess
  2. 2026How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use
  3. 2024GAMEBoT: Transparent Assessment of LLM Reasoning in Games
  4. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
  5. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  6. 2024LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
  7. 2026Clueing up LLMs with Tool-Augmented Deductive Reasoning
  8. 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injection