# Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awareness

arXiv: https://arxiv.org/abs/2509.00591 · HTML: https://mlsanity.com/paper/2509.00591/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 30 Aug 2025.

## Builds on (most cited first, 12 of 12)

- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Don't Make Your LLM an Evaluation Benchmark Cheater](https://mlsanity.com/paper/2311.01964/) (2023)
- [Taken out of context: On measuring situational awareness in LLMs](https://mlsanity.com/paper/2309.00667/) (2023)
- [Large Language Models Often Know When They Are Being Evaluated](https://mlsanity.com/paper/2505.23836/) (2025)
- [Tell me about yourself: LLMs are aware of their learned behaviors](https://mlsanity.com/paper/2501.11120/) (2025)
- [Detecting Strategic Deception Using Linear Probes](https://mlsanity.com/paper/2502.03407/) (2025)
- [Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?](https://mlsanity.com/paper/2407.21792/) (2024)
- [PRewrite: Prompt Rewriting with Reinforcement Learning](https://mlsanity.com/paper/2401.08189/) (2024)
- [S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models](https://mlsanity.com/paper/2405.14191/) (2024)
- [Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models](https://mlsanity.com/paper/2412.01784/) (2024)
- [I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBench](https://mlsanity.com/paper/2401.17882/) (2024)
- [Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness Instruction](https://mlsanity.com/paper/2502.11084/) (2025)

## Built on by (most cited first, 9 of 9)

- [Natural Emergent Misalignment from Reward Hacking in Production RL](https://mlsanity.com/paper/2511.18397/) (2025)
- [Evaluation Awareness in Language Models Has Limited Effect on Behaviour](https://mlsanity.com/paper/2605.05835/) (2026)
- [Models That Know How Evaluations Are Designed Score Safer](https://mlsanity.com/paper/2605.28591/) (2026)
- [Evaluation Awareness Is Not One Capability: Evidence from Open Language Models](https://mlsanity.com/paper/2606.23583/) (2026)
- [SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations](https://mlsanity.com/paper/2605.22564/) (2026)
- [Decomposing and Steering Functional Metacognition in Large Language Models](https://mlsanity.com/paper/2605.08942/) (2026)
- [Evaluation Awareness in Language Models: Representation, Verbalization, and Control](https://mlsanity.com/paper/2608.21766/) (2026)
- [VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design](https://mlsanity.com/paper/2607.18181/) (2026)
- [Self-Propagating Misalignment in LLM Agents, and Why Auditing or Disabling Memory Is Not Enough](https://mlsanity.com/paper/2610.04083/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) often exhibit significant behavioral shifts when they perceive a change from a real-world deployment context to a controlled evaluation setting, a phenomenon known as "evaluation awareness." This discrepancy poses a critical challenge for AI alignment, as benchmark performance may not accurately reflect a model's true safety and honesty. In this work, we systematically quantify these behavioral changes by manipulating the perceived context of prompts. We introduce a methodology that uses a linear probe to score prompts on a continuous scale from "test-like" to "deploy-like" and leverage an LLM rewriting strategy to shift these prompts towards a more natural, deployment-style context while preserving the original task. Using this method, we achieved a 30% increase in the average probe score across a strategic role-playing dataset after rewriting. Evaluating a suite of state-of-the-art models on these original and rewritten prompts, we find that rewritten "deploy-like" prompts induce a significant and consistent shift in behavior. Across all models, we observed an average increase in honest responses of 5.26% and a corresponding average decrease in deceptive responses of 12.40%. Furthermore, refusal rates increased by an average of 6.38%, indicating heightened safety compliance. Our findings demonstrate that evaluation awareness is a quantifiable and manipulable factor that directly influences LLM behavior, revealing that models are more prone to unsafe or deceptive outputs in perceived test environments. This underscores the urgent need for more realistic evaluation frameworks to accurately gauge true model alignment before deployment.