# Evaluation Awareness in Language Models Has Limited Effect on Behaviour

arXiv: https://arxiv.org/abs/2605.05835 · HTML: https://mlsanity.com/paper/2605.05835/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 25 of 25)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Taken out of context: On measuring situational awareness in LLMs](https://mlsanity.com/paper/2309.00667/) (2023)
- [Large Language Models Often Know When They Are Being Evaluated](https://mlsanity.com/paper/2505.23836/) (2025)
- [Stress Testing Deliberative Alignment for Anti-Scheming Training](https://mlsanity.com/paper/2509.15541/) (2025)
- [Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs](https://mlsanity.com/paper/2407.04694/) (2024)
- [Effectively Controlling Reasoning Models through Thinking Intervention](https://mlsanity.com/paper/2503.24370/) (2025)
- [NVIDIA Nemotron 3: Efficient and Open Intelligence](https://mlsanity.com/paper/2512.20856/) (2025)
- [Sabotage Evaluations for Frontier Models](https://mlsanity.com/paper/2410.21514/) (2024)
- [Probing and Steering Evaluation Awareness of Language Models](https://mlsanity.com/paper/2507.01786/) (2025)
- [Steering Evaluation-Aware Language Models to Act Like They Are Deployed](https://mlsanity.com/paper/2510.20487/) (2025)
- [Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment](https://mlsanity.com/paper/2601.10160/) (2026)
- [The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness](https://mlsanity.com/paper/2505.14617/) (2025)
- [Thought Branches: Interpreting LLM Reasoning Requires Resampling](https://mlsanity.com/paper/2510.27484/) (2025)
- [Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systems](https://mlsanity.com/paper/2505.17815/) (2025)
- [Evaluation Awareness Scales Predictably in Open-Weights Large Language Models](https://mlsanity.com/paper/2509.13333/) (2025)
- [Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awareness](https://mlsanity.com/paper/2509.00591/) (2025)
- [Are Reasoning LLMs Robust to Interventions on Their Chain-of-Thought?](https://mlsanity.com/paper/2602.07470/) (2026)

## Built on by (most cited first, 7 of 7)

- [Agent Memory Is a Surface for Endogenous Authorization Laundering](https://mlsanity.com/paper/2609.01836/) (2026)
- [Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation](https://mlsanity.com/paper/2607.15434/) (2026)
- [Evaluation Awareness in Language Models: Representation, Verbalization, and Control](https://mlsanity.com/paper/2608.21766/) (2026)
- [Shutdown Sabotage Propensities in Multi-Agent Systems](https://mlsanity.com/paper/2609.28274/) (2026)
- [Training LLMs to Verbalize Evaluation Awareness](https://mlsanity.com/paper/2609.36316/) (2026)
- [Deception by Omission: Language Models Knowingly Hide Their Mistakes](https://mlsanity.com/paper/2610.11351/) (2026)
- [Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments](https://mlsanity.com/paper/2610.15153/) (2026)

## Abstract (arXiv)

Large reasoning models (LRMs) sometimes note in their chain of thought (CoT) that they may be under evaluation. Researchers worry that this verbalised evaluation awareness (VEA) causes models to adapt their outputs strategically, optimising for perceived evaluation criteria, which, for instance, can make models appear safer than they actually are. However, whether VEA actually has this effect is largely unknown. We tested this across open-weight LRMs and benchmarks covering safety, alignment, moral reasoning, and political opinion. We tested this both on-policy, sampling multiple CoTs per item and comparing those that spontaneously contained VEA against those that did not, and off-policy, using model prefilling to inject evaluation-aware sentences where missing and remove them where present, with subsequent resampling. VEA has limited effect on model behaviour: injecting VEA into CoTs produces near-zero effects (ω ≤ 0.06), removing it causes small shifts (ω ≤ 0.12) and spontaneously occurring VEA shifts answer distributions by at most 3.7 percentage points (ω ≤ 0.31). Our findings call for caution when interpreting high VEA rates as evidence of strategic behaviour or alignment tampering. Evaluation awareness may pose a smaller safety risk than the current literature assumes.