mlsanityData to 7 Oct 2026

Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures

arXiv 2603.16475 ↗First posted 17 Mar 2026cs.AIcited by 5

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

In schema-guided reasoning (SGR) pipelines, LLMs produce explicit intermediate structures -- rubrics, checklists, or verification queries -- before committing to a final decision. SGR is increasingly adopted because it promises controllability: practitioners expect to inspect, edit, and override these structures to steer the outcome. But does the promise hold? We introduce a causal evaluation protocol to measure it: by selecting tasks where a deterministic function maps intermediate structures to decisions, every controlled edit implies a unique correct output. Across 12 models and 4 benchmarks, models appear self-consistent with their own intermediate structures but fail to update predictions after intervention -- revealing that apparent faithfulness is fragile once the intermediate structure changes. When derivation of the final decision from the structure is delegated to an external tool, this fragility largely disappears; stronger prompting yields only limited improvements, while preference optimization substantially improves intervention faithfulness. Overall, intermediate structures in schema-guided pipelines function as influential context rather than stable causal mediators.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 36 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

22 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
  2. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps20 Feb
  3. 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanations13 Nov
  4. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May

Other references · 18

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2024A Survey on Large Language Models for Code Generationcited by 646
  3. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  4. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  5. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  6. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  7. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  8. 2013Direct and Indirect Effectscited by 276
  9. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  10. 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 178
  11. 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
  12. 2024MedCalc-Bench: Evaluating Large Language Models for Medical Calculationscited by 64
6 more
  1. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 63
  2. 2024"We Need Structured Output": Towards User-centered Constraints on Large Language Model Outputcited by 46
  3. 2024Monitoring Latent World States in Language Models with Propositional Probescited by 39
  4. 2024Automated Long Answer Grading with RiceChem Datasetcited by 10
  5. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Modelscited by 9
  6. 2025Structured Prompting and Feedback-Guided Reasoning with LLMs for Data Interpretationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning
  2. 2026C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning
  3. 2026Compliance vs. Sensibility: On the Reasoning Controllability in Large Language Models
  4. 2026Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters
  5. 2025Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models
  6. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
  7. 2026Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents
  8. 2026Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models