# Monitoring Monitorability

arXiv: https://arxiv.org/abs/2512.18311 · HTML: https://mlsanity.com/paper/2512.18311/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- First posted on arXiv 20 Dec 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Built on by (most cited first, 30 of 58)

- [Reasoning Models Struggle to Control their Chains of Thought](https://mlsanity.com/paper/2603.05706/) (2026)
- [Counterfactual Simulation Training for Chain-of-Thought Faithfulness](https://mlsanity.com/paper/2602.20710/) (2026)
- [When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents](https://mlsanity.com/paper/2602.08235/) (2026)
- [Quantifying the Necessity of Chain of Thought through Opaque Serial Depth](https://mlsanity.com/paper/2603.09786/) (2026)
- [Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures](https://mlsanity.com/paper/2609.29429/) (2026)
- [Probing the Trajectories of Reasoning Traces in Large Language Models](https://mlsanity.com/paper/2601.23163/) (2026)
- [Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning](https://mlsanity.com/paper/2604.22074/) (2026)
- [Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation](https://mlsanity.com/paper/2601.14691/) (2026)
- [Training Agents to Self-Report Misbehavior](https://mlsanity.com/paper/2602.22303/) (2026)
- [Do explanations generalize across large reasoning models?](https://mlsanity.com/paper/2601.11517/) (2026)
- [Chain-of-thought obfuscation learned from output supervision can generalise to unseen tasks](https://mlsanity.com/paper/2601.23086/) (2026)
- [Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought?](https://mlsanity.com/paper/2603.30036/) (2026)
- [ReasonOps: Operator Segmentation for LLM Reasoning Traces](https://mlsanity.com/paper/2605.29192/) (2026)
- [LEDGER: Claim-to-Evidence Trace Graphs for Auditing LLM Agents](https://mlsanity.com/paper/2608.18398/) (2026)
- [Automated Researchers Can Mitigate Well-characterized Alignment Failures](https://mlsanity.com/paper/2608.28945/) (2026)
- [Monitorability as a Free Gift: How RLVR Spontaneously Aligns Reasoning](https://mlsanity.com/paper/2602.03978/) (2026)
- [Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models](https://mlsanity.com/paper/2603.26410/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)
- [Reinforcement Learning Towards Broadly and Persistently Beneficial Models](https://mlsanity.com/paper/2606.24014/) (2026)
- [An Independent Safety Evaluation of Kimi K2.5](https://mlsanity.com/paper/2604.03121/) (2026)
- [Value-Conflict Diagnostics Reveal Widespread Alignment Faking in Language Models](https://mlsanity.com/paper/2604.20995/) (2026)
- [Formal Methods Meet LLMs: Auditing, Monitoring, and Intervention for Compliance of Advanced AI Systems](https://mlsanity.com/paper/2605.16198/) (2026)
- [Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2605.24286/) (2026)
- [How Transparent is DiffusionGemma?](https://mlsanity.com/paper/2606.20560/) (2026)
- [Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values](https://mlsanity.com/paper/2607.14345/) (2026)
- [AI Finds A Way](https://mlsanity.com/paper/2608.23875/) (2026)
- [Diagnosing Pathological Chain-of-Thought in Reasoning Models](https://mlsanity.com/paper/2602.13904/) (2026)
- [The Detection-Extraction Gap: Models Know the Answer Before They Can Say It](https://mlsanity.com/paper/2604.06613/) (2026)
- [Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings](https://mlsanity.com/paper/2608.04735/) (2026)
- [Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations](https://mlsanity.com/paper/2609.19101/) (2026)

## Abstract (arXiv)

Observability into the decision making of modern AI systems may be required to safely deploy increasingly capable agents. Monitoring the chain-of-thought (CoT) of today's reasoning models has proven effective for detecting misbehavior. However, this "monitorability" may be fragile under different training procedures, data sources, or even continued system scaling. To measure and track monitorability, we propose three evaluation archetypes (intervention, process, and outcome-property) and a new monitorability metric, and introduce a broad evaluation suite. We demonstrate that these evaluations can catch simple model organisms trained to have obfuscated CoTs, and that CoT monitoring is more effective than action-only monitoring in practical settings. We compare the monitorability of various frontier models and find that most models are fairly, but not perfectly, monitorable. We also evaluate how monitorability scales with inference-time compute, reinforcement learning optimization, and pre-training model size. We find that longer CoTs are generally more monitorable and that RL optimization does not materially decrease monitorability even at the current frontier scale. Notably, we find that for a model at a low reasoning effort, we could instead deploy a smaller model at a higher reasoning effort (thereby matching capabilities) and obtain a higher monitorability, albeit at a higher overall inference compute cost. We further investigate agent-monitor scaling trends and find that scaling a weak monitor's test-time compute when monitoring a strong agent increases monitorability. Giving the weak monitor access to CoT not only improves monitorability, but it steepens the monitor's test-time compute to monitorability scaling trend. Finally, we show we can improve monitorability by asking models follow-up questions and giving their follow-up CoT to the monitor.