mlsanityData to 7 Oct 2026

Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing

arXiv 2603.17199 ↗First posted 17 Mar 2026cs.LG cs.AI cs.CLcited by 6

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) can produce chains of thought (CoT) that do not accurately reflect the actual factors driving their answers. In multiple-choice settings with an injected hint favoring a particular option, models may shift their final answer toward the hinted option and produce a CoT that rationalizes the response without acknowledging the hint - an instance of motivated reasoning. We study this phenomenon across multiple LLM families and datasets demonstrating that motivated reasoning can be identified by probing internal activations even in cases when it cannot be easily determined from CoT. Using supervised probes trained on the model's residual stream, we show that (i) pre-generation probes, applied before any CoT tokens are generated, predict motivated reasoning as well as a LLM-based CoT monitor that accesses the full CoT trace, and (ii) post-generation probes, applied after CoT generation, outperform the same monitor. Together, these results show that motivated reasoning is detected more reliably from internal representations than from CoT monitoring. Moreover, pre-generation probing can flag motivated behavior early, potentially avoiding unnecessary generation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 49 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

26 of 49 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026Sycophantic Anchors: Localizing and Quantifying User Agreement in Reasoning Models29 Jan
  2. 2025The Ends Justify the Thoughts: RL-Induced Motivated Reasoning in LLM CoTs20 Oct
  3. 2025Thought Anchors: Which LLM Reasoning Steps Matter?23 Jun
  4. 2025Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs20 May
  5. 2025Reasoning Models Don't Always Say What They Think8 May
  6. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
  7. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
  8. 2024LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics2 Dec
  9. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models15 Jun
  10. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul

Other references · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2025Gemma 3 Technical Reportcited by 1,686
  5. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  6. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  7. 2025Open Problems in Mechanistic Interpretabilitycited by 173
  8. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 69
  9. 2025Detecting Strategic Deception Using Linear Probescited by 65
  10. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  11. 2024Do language models plan ahead for future tokens?cited by 47
  12. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 35
4 more
  1. 2025Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Onescited by 12
  2. 2025Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoningcited by 12
  3. 2025MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Modelscited by 10
  4. 2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026How Much Does a Reasoning Summary Reveal? An Observability Ladder for Large Language Modelscited by 1
  2. 2026Forecasting Future Behavior as a Learning Taskcited by 0
  3. 2026Hyperparameter Transfer in Graph Neural Networkscited by 0
  4. 2026Committed Before Reasoning: Behavioral Reproduction and Preliminary Activation-Level Evidence of Answer Pre-Commitment in an Open-Weight LLMcited by 0
  5. 2026Two Regimes of Chain-of-Thought Unfaithfulness: Metric-Based Detection Fails Where Models Are Wrongcited by 0
  6. 2026Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models
  2. 2026Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators
  3. 2026Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamics
  4. 2024Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step
  5. 2026Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
  6. 2025Lost at the Beginning of Reasoning
  7. 2025The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think
  8. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation