From Decorative to Load-Bearing: Task Difficulty Shapes the Causal Role of Chain-of-Thought
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
Chain-of-thought (CoT) monitoring is only meaningful if written reasoning causally constrains the answer. We introduce continuation-based causal testing, an ablation-patch intervention that perturbs one reasoning step, truncates the chain, and forces the model to continue from the corrupted prefix. It measures how load-bearing a CoT is for the final answer, a behavioral notion distinct from mechanistic faithfulness. Across Gemma-2-9B-IT, Llama-3.1-8B-Instruct, and DeepSeek-R1-Distill-Qwen-7B on GSM8K, MMLU, and BIG-Bench Hard, CoT load-bearingness tracks model-relative task difficulty: on easy tasks models silently bypass their own reasoning; on hard tasks they follow corrupted steps and propagate errors. A matched 2x2 analysis shows task difficulty dominates perturbation type: error propagation rises 16x from GSM8K to BBH multistep arithmetic, and a variance partition over 28,584 continuations attributes 98.8% of explained deviance to task difficulty versus 0.8% to perturbation type. Reasoning-specific RL suppresses error propagation and compresses the gradient. A four-variant judge-sensitivity analysis and blind two-annotator study (n=500) show the error-propagation vs. non-propagation label is invariant to judge prompt, with perfect inter-annotator agreement (Cohen's kappa = 1.00). This gradient creates a structural problem for CoT-based oversight and AI safety monitoring: where the trace is easy to read it carries little signal, and where it matters errors propagate before a monitor can intervene. Linear probes on hidden states separate silent bypass, self-correction, and error propagation, but additive activation steering provides limited causal control, flipping only about 25% of error-propagation cases at best. Behavioral mode is readable but not reliably controllable.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 25 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
9 of 25 references.
Closest ancestors · 6
Same direction, newest first
- 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding24 Jul
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety15 Jul
- 2025Reasoning Models Don't Always Say What They Think8 May
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
- 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models15 Jun
- 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
Other references · 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
- 2026Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning
- 2026When Chain-of-Thought Fails, the Solution Hides in the Hidden States
- 2026Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings
- 2026Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models
- 2025Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
- 2026Are Stated Reasoning Steps Causally Load-Bearing?
- 2026When does Chain-of-Thought Help: A Markovian Perspective