# Realistic honeypot evaluations for scheming propensity

arXiv: https://arxiv.org/abs/2605.29729 · HTML: https://mlsanity.com/paper/2605.29729/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 21 of 21)

- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Agentic Misalignment: How LLMs Could Be Insider Threats](https://mlsanity.com/paper/2510.05179/) (2025)
- [Large Language Models can Strategically Deceive their Users when Put Under Pressure](https://mlsanity.com/paper/2311.07590/) (2023)
- [Evaluating Frontier Models for Dangerous Capabilities](https://mlsanity.com/paper/2403.13793/) (2024)
- [Large Language Models Often Know When They Are Being Evaluated](https://mlsanity.com/paper/2505.23836/) (2025)
- [Is Power-Seeking AI an Existential Risk?](https://mlsanity.com/paper/2206.13353/) (2022)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [Stress Testing Deliberative Alignment for Anti-Scheming Training](https://mlsanity.com/paper/2509.15541/) (2025)
- [Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs](https://mlsanity.com/paper/2407.04694/) (2024)
- [Sabotage Evaluations for Frontier Models](https://mlsanity.com/paper/2410.21514/) (2024)
- [Evaluating Frontier Models for Stealth and Situational Awareness](https://mlsanity.com/paper/2505.01420/) (2025)
- [The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness](https://mlsanity.com/paper/2505.14617/) (2025)
- [Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs](https://mlsanity.com/paper/2509.14260/) (2025)
- [A Game-Theoretic Taxonomy and Survey of Defensive Deception for Cybersecurity and Privacy](https://mlsanity.com/paper/1712.05441/) (2017)
- [Evaluating and Understanding Scheming Propensity in LLM Agents](https://mlsanity.com/paper/2603.01608/) (2026)
- [LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wild](https://mlsanity.com/paper/2410.13919/) (2024)
- [Evaluating whether AI models would sabotage AI safety research](https://mlsanity.com/paper/2604.24618/) (2026)
- [Propensity Inference: Environmental Contributors to LLM Behaviour](https://mlsanity.com/paper/2604.21098/) (2026)
- [Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors](https://mlsanity.com/paper/2605.06490/) (2026)
- [MISR: Measuring Instrumental Self-Reasoning in Frontier Models](https://mlsanity.com/paper/2412.03904/) (2024)

## Built on by (most cited first, 1 of 1)

- [Mechanism Design for Alignment and Control](https://mlsanity.com/paper/2609.01595/) (2026)

## Abstract (arXiv)

We introduce scheming honeypot evaluations, a framework for testing whether models will pursue instrumental goals if given the opportunity. Our scheming honeypot evaluations take the form of coding tasks in Google's alignment research codebases. In a real internal deployment setting, Gemini models do not demonstrate unprompted scheming. If prompts explicitly encourage agency (situational awareness or goal-directedness) and/or give the model a hidden goal, models sometimes scheme or attempt sabotage. Validating the realism of our setting, models show low rates of evaluation awareness, usually due to agency prompts rather than the environments.