# Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity

arXiv: https://arxiv.org/abs/2510.27378 · HTML: https://mlsanity.com/paper/2510.27378/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 29 of its 67 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 29 of 29)

- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [Large Language Models can Strategically Deceive their Users when Put Under Pressure](https://mlsanity.com/paper/2311.07590/) (2023)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2402.13950/) (2024)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [Preventing Language Models From Hiding Their Reasoning](https://mlsanity.com/paper/2310.18512/) (2023)
- [CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring](https://mlsanity.com/paper/2505.23575/) (2025)
- [Inverse Scaling in Test-Time Compute](https://mlsanity.com/paper/2507.14417/) (2025)
- [Large language models can learn and generalize steganographic chain-of-thought under process supervision](https://mlsanity.com/paper/2506.01926/) (2025)
- [Base Models Know How to Reason, Thinking Models Learn When](https://mlsanity.com/paper/2510.07364/) (2025)
- [Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.22777/) (2025)
- [Early Signs of Steganographic Capabilities in Frontier LLMs](https://mlsanity.com/paper/2507.02737/) (2025)
- [Robustly Improving LLM Fairness in Realistic Settings via Interpretability](https://mlsanity.com/paper/2506.10922/) (2025)
- [LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring](https://mlsanity.com/paper/2508.00943/) (2025)
- [The Steganographic Potentials of Language Models](https://mlsanity.com/paper/2505.03439/) (2025)

## Built on by (most cited first, 17 of 17)

- [Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?](https://mlsanity.com/paper/2603.22582/) (2026)
- [Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluation](https://mlsanity.com/paper/2603.20172/) (2026)
- [Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models](https://mlsanity.com/paper/2603.26410/) (2026)
- [Decomposing and Measuring Evaluation Awareness](https://mlsanity.com/paper/2605.23055/) (2026)
- [When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel](https://mlsanity.com/paper/2605.11746/) (2026)
- [Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression](https://mlsanity.com/paper/2608.01631/) (2026)
- [Conversable Complexity: Agentic LLM Collectives as Interpretable Substrates](https://mlsanity.com/paper/2607.01047/) (2026)
- [Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered](https://mlsanity.com/paper/2608.29464/) (2026)
- [CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News Generation](https://mlsanity.com/paper/2602.04856/) (2026)
- [Risk Reporting for Developers' Internal AI Model Use](https://mlsanity.com/paper/2604.24966/) (2026)
- [What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation](https://mlsanity.com/paper/2605.26795/) (2026)
- [On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness](https://mlsanity.com/paper/2607.29062/) (2026)
- [Untangling the Mechanisms of Misleading Context in Medical Question Answering](https://mlsanity.com/paper/2609.02754/) (2026)
- [Do Frontier Models Seek Safety Evidence Before Acting?](https://mlsanity.com/paper/2609.17865/) (2026)
- [Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcement](https://mlsanity.com/paper/2609.24243/) (2026)
- [Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using Agents](https://mlsanity.com/paper/2609.27542/) (2026)
- [When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Making](https://mlsanity.com/paper/2610.19948/) (2026)

## Abstract (arXiv)

Chain-of-thought (CoT) outputs let us read a model's step-by-step reasoning. Since any long, serial reasoning process must pass through this textual trace, the quality of the CoT is a direct window into what the model is thinking. This visibility could help us spot unsafe or misaligned behavior (monitorability), but only if the CoT is transparent about its internal reasoning (faithfulness). Fully measuring faithfulness is difficult, so researchers often focus on examining the CoT in cases where the model changes its answer after adding a cue to the input. This proxy finds some instances of unfaithfulness but loses information when the model maintains its answer, and does not investigate aspects of reasoning not tied to the cue. We extend these results to a more holistic sense of monitorability by introducing verbosity: whether the CoT lists every factor needed to solve the task. We combine faithfulness and verbosity into a single monitorability score that shows how well the CoT serves as the model's external `working memory', a property that many safety schemes based on CoT monitoring depend on. We evaluate instruction-tuned and reasoning models on BBH, GPQA, and MMLU. Our results show that models can appear faithful yet remain hard to monitor when they leave out key factors, and that monitorability differs sharply across model families. We release our evaluation code using the Inspect library to support reproducible future work.