mlsanityData to 14 Oct 2026

Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity

arXiv 2510.27378 ↗First posted 31 Oct 2025cs.LG cs.AI cs.CLcited by 17

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) outputs let us read a model's step-by-step reasoning. Since any long, serial reasoning process must pass through this textual trace, the quality of the CoT is a direct window into what the model is thinking. This visibility could help us spot unsafe or misaligned behavior (monitorability), but only if the CoT is transparent about its internal reasoning (faithfulness). Fully measuring faithfulness is difficult, so researchers often focus on examining the CoT in cases where the model changes its answer after adding a cue to the input. This proxy finds some instances of unfaithfulness but loses information when the model maintains its answer, and does not investigate aspects of reasoning not tied to the cue. We extend these results to a more holistic sense of monitorability by introducing verbosity: whether the CoT lists every factor needed to solve the task. We combine faithfulness and verbosity into a single monitorability score that shows how well the CoT serves as the model's external `working memory', a property that many safety schemes based on CoT monitoring depend on. We evaluate instruction-tuned and reasoning models on BBH, GPQA, and MMLU. Our results show that models can appear faithful yet remain hard to monitor when they leave out key factors, and that monitorability differs sharply across model families. We release our evaluation code using the Inspect library to support reproducible future work.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 67 references that are in the map. First posted on arXiv 31 Oct 2025.

Builds on

29 of 67 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring31 Jul
  2. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety15 Jul
  3. 2025When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors7 Jul
  4. 2025Large language models can learn and generalize steganographic chain-of-thought under process supervision2 Jun
  5. 2025CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring29 May
  6. 2025Reasoning Models Don't Always Say What They Think8 May
  7. 2025The Steganographic Potentials of Language Models6 May
  8. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
  9. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
  10. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning21 Feb
  11. 2023Preventing Language Models From Hiding Their Reasoning27 Oct
  12. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
2 more
  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
  2. 2022Large Language Models are Zero-Shot Reasoners24 May

Other references · 15

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2024Frontier Models are Capable of In-context Schemingcited by 287
  7. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  8. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
  9. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
  10. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  11. 2025Inverse Scaling in Test-Time Computecited by 50
  12. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 27
3 more
  1. 2025Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoningcited by 21
  2. 2025Early Signs of Steganographic Capabilities in Frontier LLMscited by 19
  3. 2025Robustly Improving LLM Fairness in Realistic Settings via Interpretabilitycited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 17.

17 papers.

Sort:

By area:

  1. 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?cited by 8
  2. 2026Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluationcited by 4
  3. 2026Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Modelscited by 4
  4. 2026Decomposing and Measuring Evaluation Awarenesscited by 3
  5. 2026When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channelcited by 2
  6. 2026Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compressioncited by 2
  7. 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substratescited by 1
  8. 2026Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Deliveredcited by 1
  9. 2026CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News Generationcited by 0
  10. 2026Risk Reporting for Developers' Internal AI Model Usecited by 0
  11. 2026What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activationcited by 0
  12. 2026On the Generalization of Steering Vectors for Chain-of-Thought Faithfulnesscited by 0
5 more
  1. 2026Untangling the Mechanisms of Misleading Context in Medical Question Answeringcited by 0
  2. 2026Do Frontier Models Seek Safety Evidence Before Acting?cited by 0
  3. 2026Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcementcited by 0
  4. 2026Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using Agentscited by 0
  5. 2026When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Makingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?
  2. 2026Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
  3. 2025Investigating CoT Monitorability in Large Reasoning Models
  4. 2026MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
  5. 2025A Pragmatic Way to Measure Chain-of-Thought Monitorability
  6. 2026Reasoning Models Struggle to Control their Chains of Thought
  7. 2026Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models
  8. 2026Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings