mlsanityData to 7 Oct 2026

From Decorative to Load-Bearing: Task Difficulty Shapes the Causal Role of Chain-of-Thought

arXiv 2609.25366 ↗First posted 21 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) monitoring is only meaningful if written reasoning causally constrains the answer. We introduce continuation-based causal testing, an ablation-patch intervention that perturbs one reasoning step, truncates the chain, and forces the model to continue from the corrupted prefix. It measures how load-bearing a CoT is for the final answer, a behavioral notion distinct from mechanistic faithfulness. Across Gemma-2-9B-IT, Llama-3.1-8B-Instruct, and DeepSeek-R1-Distill-Qwen-7B on GSM8K, MMLU, and BIG-Bench Hard, CoT load-bearingness tracks model-relative task difficulty: on easy tasks models silently bypass their own reasoning; on hard tasks they follow corrupted steps and propagate errors. A matched 2x2 analysis shows task difficulty dominates perturbation type: error propagation rises 16x from GSM8K to BBH multistep arithmetic, and a variance partition over 28,584 continuations attributes 98.8% of explained deviance to task difficulty versus 0.8% to perturbation type. Reasoning-specific RL suppresses error propagation and compresses the gradient. A four-variant judge-sensitivity analysis and blind two-annotator study (n=500) show the error-propagation vs. non-propagation label is invariant to judge prompt, with perfect inter-annotator agreement (Cohen's kappa = 1.00). This gradient creates a structural problem for CoT-based oversight and AI safety monitoring: where the trace is easy to read it carries little signal, and where it matters errors propagate before a monitor can intervene. Linear probes on hidden states separate silent bypass, self-correction, and error propagation, but additive activation steering provides limited causal control, flipping only about 25% of error-propagation cases at best. Behavioral mode is readable but not reliably controllable.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 25 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

9 of 25 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding24 Jul
  2. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety15 Jul
  3. 2025Reasoning Models Don't Always Say What They Think8 May
  4. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
  5. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models15 Jun
  6. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul

Other references · 3

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 107
  3. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 69

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
  2. 2026Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning
  3. 2026When Chain-of-Thought Fails, the Solution Hides in the Hidden States
  4. 2026Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings
  5. 2026Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models
  6. 2025Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
  7. 2026Are Stated Reasoning Steps Causally Load-Bearing?
  8. 2026When does Chain-of-Thought Help: A Markovian Perspective