# Reliable Weak-to-Strong Monitoring of LLM Agents

arXiv: https://arxiv.org/abs/2508.19461 · HTML: https://mlsanity.com/paper/2508.19461/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 50 of its 123 references that are in the map.
- First posted on arXiv 26 Aug 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 30 of 50)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails](https://mlsanity.com/paper/2310.10501/) (2023)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)

## Built on by (most cited first, 30 of 31)

- [Async Control: Stress-testing Asynchronous Control Measures for LLM Agents](https://mlsanity.com/paper/2512.13526/) (2025)
- [A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes](https://mlsanity.com/paper/2601.05293/) (2026)
- [Detecting Safety Violations Across Many Agent Traces](https://mlsanity.com/paper/2604.11806/) (2026)
- [AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems](https://mlsanity.com/paper/2605.08715/) (2026)
- [Peer-Preservation in Frontier Models](https://mlsanity.com/paper/2604.19784/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures](https://mlsanity.com/paper/2609.29429/) (2026)
- [Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs](https://mlsanity.com/paper/2602.08563/) (2026)
- [LHAW: Controllable Underspecification for Long-Horizon Tasks](https://mlsanity.com/paper/2602.10525/) (2026)
- [Training Agents to Self-Report Misbehavior](https://mlsanity.com/paper/2602.22303/) (2026)
- [SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors](https://mlsanity.com/paper/2605.16626/) (2026)
- [Automated Researchers Can Mitigate Well-characterized Alignment Failures](https://mlsanity.com/paper/2608.28945/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)
- [Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?](https://mlsanity.com/paper/2606.05647/) (2026)
- [CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs](https://mlsanity.com/paper/2606.11063/) (2026)
- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)
- [ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D](https://mlsanity.com/paper/2607.19321/) (2026)
- [Quality-Driven Agentic Reasoning for LLM-Assisted Software Design: Questions-of-Thoughts (QoT) as a Time-Series Self-QA Chain](https://mlsanity.com/paper/2603.11082/) (2026)
- [Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages](https://mlsanity.com/paper/2607.06596/) (2026)
- [Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents](https://mlsanity.com/paper/2610.01023/) (2026)
- [CoT-Guard: Small Models for Strong Monitoring](https://mlsanity.com/paper/2605.12746/) (2026)
- [Agent-ToM: Learning to Monitor Autonomous LLM Agents via Theory-of-Mind Reasoning](https://mlsanity.com/paper/2605.24216/) (2026)
- [Code Monitor Red Teaming for Public-Test-Passing Code](https://mlsanity.com/paper/2607.20852/) (2026)
- [INTENT-AS-A-TOOL Makes it Easy to Track Agentic Misalignment](https://mlsanity.com/paper/2608.27348/) (2026)
- [MOLE: Detecting Insider Threats in AI Agents](https://mlsanity.com/paper/2609.06966/) (2026)
- [The Wisdom of Artificial Deliberative Crowds](https://mlsanity.com/paper/2609.22497/) (2026)
- [HARDE: Optimizing Agent Harnesses for Runtime Risk Detection and Execution Control](https://mlsanity.com/paper/2609.38291/) (2026)
- [PyINE: A Framework for Scalable Elicitation and Oversight via Code Execution](https://mlsanity.com/paper/2610.04737/) (2026)
- [Monitorability Disposition in Large Reasoning Models](https://mlsanity.com/paper/2610.04914/) (2026)
- [Towards a Unified Misuse Monitoring Benchmark](https://mlsanity.com/paper/2610.07089/) (2026)

## Abstract (arXiv)

We stress test monitoring systems for detecting covert misbehavior in autonomous LLM agents (e.g., secretly sharing private information). To this end, we systematize a monitor red teaming (MRT) workflow that incorporates: (1) varying levels of agent and monitor situational awareness; (2) distinct adversarial strategies to evade the monitor, such as prompt injection; and (3) two datasets and environments -- SHADE-Arena for tool-calling agents and our new CUA-SHADE-Arena, which extends TheAgentCompany, for computer-use agents. We run MRT on existing LLM monitor scaffoldings, which orchestrate LLMs and parse agent trajectories, alongside a new hybrid hierarchical-sequential scaffolding proposed in this work. Our empirical results yield three key findings. First, agent awareness dominates monitor awareness: an agent's knowledge that it is being monitored substantially degrades the monitor's reliability. On the contrary, providing the monitor with more information about the agent is less helpful than expected. Second, monitor scaffolding matters more than monitor awareness: the hybrid scaffolding consistently outperforms baseline monitor scaffolding, and can enable weaker models to reliably monitor stronger agents -- a weak-to-strong scaling effect. Third, in a human-in-the-loop setting where humans discuss with the LLM monitor to get an updated judgment for the agent's behavior, targeted human oversight is most effective; escalating only pre-flagged cases to human reviewers improved the TPR by approximately 15% at FPR = 0.01. Our work establishes a standard workflow for MRT, highlighting the lack of adversarial robustness for LLMs and humans when monitoring and detecting agent misbehavior. We release code, data, and logs to spur further research.