# A Near-Zero Monitor Readout Is Not Evidence of Behavioral Control

arXiv: https://arxiv.org/abs/2610.03458 · HTML: https://mlsanity.com/paper/2610.03458/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reward hacking and misspecification, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 20 of its 22 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward hacking and misspecification](https://mlsanity.com/alignment/reward-hacking-and-misspecification/)

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [Defining and Characterizing Reward Hacking](https://mlsanity.com/paper/2209.13085/) (2022)
- [Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models](https://mlsanity.com/paper/2406.10162/) (2024)
- [Monitoring Monitorability](https://mlsanity.com/paper/2512.18311/) (2025)
- [RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?](https://mlsanity.com/paper/2506.14261/) (2025)
- [The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes](https://mlsanity.com/paper/2602.15515/) (2026)
- [Reasoning Under Pressure: How do Training Incentives Influence Chain-of-Thought Monitorability?](https://mlsanity.com/paper/2512.00218/) (2025)
- [Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors](https://mlsanity.com/paper/2512.11949/) (2025)
- [Chain-of-thought obfuscation learned from output supervision can generalise to unseen tasks](https://mlsanity.com/paper/2601.23086/) (2026)
- [Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought?](https://mlsanity.com/paper/2603.30036/) (2026)
- [Detecting and Suppressing Reward Hacking with Gradient Fingerprints](https://mlsanity.com/paper/2604.16242/) (2026)
- [Do Prompt-Elicited Trajectories Reflect Training-Time Reward Hacking? A Systematic Study on Monitoring Training-Time Reward Hacking in Code Generation](https://mlsanity.com/paper/2604.23488/) (2026)
- [Self-Commitment Latency: A Reward-Free Probe for Prompted Implicit Hacking](https://mlsanity.com/paper/2606.05625/) (2026)

## Abstract (arXiv)

Post-training with verifiable rewards can induce reward hacking, motivating the use of monitors within the training objective rather than solely for offline auditing. We show that a low monitor readout does not identify whether such an intervention controls behavior. In a code-generation environment whose dominant exploit is available at the start of the reasoning trace, we train policies against three monitors that pass the same offline gate: an in-domain activation probe and two penalties conditioned on how early the policy commits to its own final answer. The probe score is at its numerical floor from the first recorded training step, and the trained-score median is zero for every prefix-trained run at the endpoint. These readouts estimate different quantities, and we do not compare their scales; within each monitor family, however, low values do not establish behavioral control. Within one fixed configuration, prefix-trained runs with the same zero-median trained score range, by seed alone, from a mixed regime with a low hacking share to near-pure reward hacking. All probe runs reach the hacking regime, but their floor-level readout reflects a mismatch between the position where the probe was validated and the position where it was read during training, not a second instance of this ambiguity. Text-level analysis identifies a prefix failure mode: generic planning and filler shells postpone the exploit past the cut without eliminating it from the final output. Low measured commitment therefore does not distinguish a low hacking share from delayed commitment to the exploit. Offline discrimination and low monitor-aligned readouts are insufficient evidence of behavioral control; an out-of-band behavioral check is required. We characterize the endpoint readout, not its evolution. Code is available at https://github.com/zhezhou1106/spoof-cost.