# Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Models

arXiv: https://arxiv.org/abs/2507.12428 · HTML: https://mlsanity.com/paper/2507.12428/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 37 of its 62 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 30 of 37)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Deliberative Alignment: Reasoning Enables Safer Language Models](https://mlsanity.com/paper/2412.16339/) (2024)
- [OpenThoughts: Data Recipes for Reasoning Models](https://mlsanity.com/paper/2506.04178/) (2025)
- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [Measuring Association Between Labels and Free-Text Rationales](https://mlsanity.com/paper/2010.12762/) (2020)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities](https://mlsanity.com/paper/2502.12025/) (2025)
- [Thought Anchors: Which LLM Reasoning Steps Matter?](https://mlsanity.com/paper/2506.19143/) (2025)
- [Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2402.13950/) (2024)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [Persona Features Control Emergent Misalignment](https://mlsanity.com/paper/2506.19823/) (2025)
- [Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective](https://mlsanity.com/paper/2506.14965/) (2025)
- [Magistral](https://mlsanity.com/paper/2506.10910/) (2025)
- [CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring](https://mlsanity.com/paper/2505.23575/) (2025)
- [Crosslingual Reasoning through Test-Time Scaling](https://mlsanity.com/paper/2505.05408/) (2025)
- [An Approach to Technical AGI Safety and Security](https://mlsanity.com/paper/2504.01849/) (2025)
- [Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought](https://mlsanity.com/paper/2403.05518/) (2024)
- [The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness](https://mlsanity.com/paper/2505.14617/) (2025)

## Built on by (most cited first, 17 of 17)

- [Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought](https://mlsanity.com/paper/2603.05488/) (2026)
- [Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training](https://mlsanity.com/paper/2510.20956/) (2025)
- [Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?](https://mlsanity.com/paper/2510.27269/) (2025)
- [MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models](https://mlsanity.com/paper/2511.06419/) (2025)
- [Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?](https://mlsanity.com/paper/2510.06036/) (2025)
- [HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimization](https://mlsanity.com/paper/2508.04010/) (2025)
- [The Impact of Off-Policy Training Data on Probe Generalisation](https://mlsanity.com/paper/2511.17408/) (2025)
- [Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning](https://mlsanity.com/paper/2604.22074/) (2026)
- [AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research](https://mlsanity.com/paper/2511.04316/) (2025)
- [Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures](https://mlsanity.com/paper/2601.19928/) (2026)
- [OLA: Output Language Alignment in Code-Switched LLM Interactions](https://mlsanity.com/paper/2601.03589/) (2026)
- [When is Your LLM Steerable?](https://mlsanity.com/paper/2606.11599/) (2026)
- [Do Prompt-Elicited Trajectories Reflect Training-Time Reward Hacking? A Systematic Study on Monitoring Training-Time Reward Hacking in Code Generation](https://mlsanity.com/paper/2604.23488/) (2026)
- [Self-ReSET: Learning to Self-Recover from Unsafe Reasoning Trajectories](https://mlsanity.com/paper/2605.08936/) (2026)
- [Towards Effective Theory of LLMs: A Representation Learning Approach](https://mlsanity.com/paper/2605.09294/) (2026)
- [When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety](https://mlsanity.com/paper/2609.34771/) (2026)
- [Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception](https://mlsanity.com/paper/2610.12445/) (2026)

## Abstract (arXiv)

Reasoning language models improve performance on complex tasks by generating long chains of thought (CoTs), but this process can also increase harmful outputs in adversarial settings. In this work, we ask whether the long CoTs can be leveraged for predictive safety monitoring: do the reasoning traces provide early signals of final response alignment that could enable timely intervention? We evaluate a range of monitoring methods using either CoT text or activations, including highly capable large language models, fine-tuned classifiers, and humans. First, we find that a simple linear probe trained on CoT activations significantly outperforms all text-based baselines in predicting whether a final response is safe or unsafe, with an average absolute increase of 13 in F1 scores over the best-performing alternatives. CoT texts are often unfaithful and misleading, while model latents provide a more reliable predictive signal. Second, the probe can be applied to early CoT segments before the response is generated, showing that alignment signals appear before reasoning completes. Error analysis reveals that the performance gap between text classifiers and the linear probe largely stems from a subset of responses we call performative CoTs, where the reasoning consistently contradicts the final response as the CoT progresses. Our findings generalize across model sizes, families, and safety benchmarks, suggesting that lightweight probes could enable real-time safety monitoring and early intervention during generation.