# From Probe Scores to Alarm Policies: Operational Validity of Activation Monitors for Language-Model Agents

arXiv: https://arxiv.org/abs/2610.04575 · HTML: https://mlsanity.com/paper/2610.04575/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 3 Oct 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 22 of 22)

- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents](https://mlsanity.com/paper/2410.06703/) (2024)
- [Detecting High-Stakes Interactions with Activation Probes](https://mlsanity.com/paper/2506.10805/) (2025)
- [Neyman-Pearson (NP) classification algorithms and NP receiver operating characteristics (NP-ROC)](https://mlsanity.com/paper/1608.03109/) (2016)
- [False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalize](https://mlsanity.com/paper/2509.03888/) (2025)
- [Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention](https://mlsanity.com/paper/2602.03338/) (2026)
- [From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents](https://mlsanity.com/paper/2606.09863/) (2026)
- [Monitoring Emergent Reward Hacking During Generation via Internal Activations](https://mlsanity.com/paper/2603.04069/) (2026)
- [Unifying Evaluation of Machine Learning Safety Monitors](https://mlsanity.com/paper/2208.14660/) (2022)
- [AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor](https://mlsanity.com/paper/2601.05752/) (2026)
- [The Impact of Off-Policy Training Data on Probe Generalisation](https://mlsanity.com/paper/2511.17408/) (2025)
- [TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling](https://mlsanity.com/paper/2605.27690/) (2026)
- [CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation](https://mlsanity.com/paper/2604.09155/) (2026)
- [When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents](https://mlsanity.com/paper/2606.22864/) (2026)
- [A Structured Approach to Safety Case Construction for AI Systems](https://mlsanity.com/paper/2601.22773/) (2026)
- [SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors](https://mlsanity.com/paper/2605.16626/) (2026)
- [Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness](https://mlsanity.com/paper/2606.15980/) (2026)
- [CP-Agent: A Calibrated Risk-Controlled Agent for Feedback-Driven Competitive Programming](https://mlsanity.com/paper/2605.24693/) (2026)
- [From Reward-Hack Activations to Agentic Risk States: Context-Calibrated Mechanistic Monitoring in LLM Agents](https://mlsanity.com/paper/2606.06223/) (2026)
- [PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis](https://mlsanity.com/paper/2607.12624/) (2026)
- [Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations](https://mlsanity.com/paper/2605.28553/) (2026)

## Abstract (arXiv)

Activation probes can predict safety-relevant properties of language models with high area under the receiver-operating-characteristic curve (AUROC), but deployed agent monitors make thresholded alarm decisions under tight false-alarm budgets. These are different estimands. We introduce an Operational Validity Contract that fixes a monitor's target, observability, identity, timing, intervention unit, comparator, calibration, and cost. We formalize risk at the semantic request or trajectory level: when one task contains repeated alarm opportunities, row-level AUROC and false positive rate do not identify semantic-unit any-alarm risk. A confidence-certified threshold also requires enough independent negative units, a tie-safe rule, and transport to deployment. Across Models Under Pressure, LASR refusal prediction, immutable AgentDojo, and a prospectively protocol-frozen ST-WebAgentBench replication, joint activation-observable monitors reach AUROC 0.957 and 0.935 on the first two benchmarks, yet their locked 5%/10% detection rates are only .642/.742 and .719/.782, respectively. On AgentDojo, the secondary mean-activation rollout monitor reaches AUROC 0.922 but detects none of 38 positive semantic cases at the locked 5% operating point; thresholds intended for 10% false alarms realize 18.7-20.0% on test. Because the test misses its prospectively frozen 40-positive support gate, we label it support-insufficient. On ST-WebAgentBench, activation reaches AUROC .874, but 23 independent calibration negatives cannot identify even a 10% controller; the locked policy abstains rather than reporting its mechanical zero FPR as a success. An exploratory counterexample also lowers full AUROC while improving realized 10% utility. The fail-closed compiler caps MUP and LASR at restricted predictive value and AgentDojo and ST-Web at representation accessibility; no setting reaches alarm-policy validity.