# Counterfactual Simulation Training for Chain-of-Thought Faithfulness

arXiv: https://arxiv.org/abs/2602.20710 · HTML: https://mlsanity.com/paper/2602.20710/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 45 of its 55 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 30 of 45)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?](https://mlsanity.com/paper/2005.01831/) (2020)
- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [Measuring Association Between Labels and Free-Text Rationales](https://mlsanity.com/paper/2010.12762/) (2020)
- [Looking Inward: Language Models Can Learn About Themselves by Introspection](https://mlsanity.com/paper/2410.13787/) (2024)
- [Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2402.13950/) (2024)
- [Question Decomposition Improves the Faithfulness of Model-Generated Reasoning](https://mlsanity.com/paper/2307.11768/) (2023)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?](https://mlsanity.com/paper/2010.04119/) (2020)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations](https://mlsanity.com/paper/2307.08678/) (2023)
- [Are DeepSeek R1 And Other Reasoning Models More Faithful?](https://mlsanity.com/paper/2501.08156/) (2025)
- [Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations](https://mlsanity.com/paper/1910.03065/) (2019)
- [Monitoring Monitorability](https://mlsanity.com/paper/2512.18311/) (2025)
- [CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring](https://mlsanity.com/paper/2505.23575/) (2025)

## Built on by (most cited first, 17 of 17)

- [SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancy](https://mlsanity.com/paper/2604.02423/) (2026)
- [Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision](https://mlsanity.com/paper/2606.32038/) (2026)
- [Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2605.24286/) (2026)
- [Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values](https://mlsanity.com/paper/2607.14345/) (2026)
- [When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel](https://mlsanity.com/paper/2605.11746/) (2026)
- [Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments](https://mlsanity.com/paper/2608.16747/) (2026)
- [Measuring and curing reasoning rigidity: from decorative chain-of-thought to genuine faithfulness](https://mlsanity.com/paper/2603.22816/) (2026)
- [When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't](https://mlsanity.com/paper/2604.06422/) (2026)
- [Self-CTRL: Self-Consistency Training with Reinforcement Learning](https://mlsanity.com/paper/2606.18327/) (2026)
- [Position: It's Time to Optimize LLMs for Self-Consistency](https://mlsanity.com/paper/2608.05188/) (2026)
- [Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered](https://mlsanity.com/paper/2608.29464/) (2026)
- [Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality](https://mlsanity.com/paper/2604.04418/) (2026)
- [Risky Business: Measuring The Faithfulness-Safety Tension](https://mlsanity.com/paper/2608.03745/) (2026)
- [A Removal Based Approach to Improve LLM Faithfulness at Test-Time](https://mlsanity.com/paper/2609.04343/) (2026)
- [Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcement](https://mlsanity.com/paper/2609.24243/) (2026)
- [When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment](https://mlsanity.com/paper/2609.35074/) (2026)
- [Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment](https://mlsanity.com/paper/2609.38972/) (2026)

## Abstract (arXiv)

Inspecting Chain-of-Thought reasoning is among the most common means of understanding why an LLM produced its output. But well-known problems with CoT faithfulness severely limit what insights can be gained from this practice. In this paper, we introduce a training method called Counterfactual Simulation Training (CST), which aims to improve CoT faithfulness by rewarding CoTs that enable a simulator to accurately predict a model's outputs over counterfactual inputs. We apply CST in two settings: (1) CoT monitoring with cue-based counterfactuals, to detect when models rely on spurious features, reward hack, or are sycophantic, and (2) counterfactual simulation over generic model-based counterfactuals, to encourage models to produce more faithful, generalizable reasoning in the CoT. Experiments with models up to 235B parameters show that CST can substantially improve monitor accuracy on cue-based counterfactuals (by 35 accuracy points) as well as simulatability over generic counterfactuals (by 2 points). We further show that: (1) CST outperforms prompting baselines, (2) rewriting unfaithful CoTs with an LLM is 5x more efficient than RL alone, (3) faithfulness improvements do not generalize to dissuading cues (as opposed to persuading cues), and (4) larger models do not show more faithful CoT out of the box, but they do benefit more from CST. These results suggest that CST can improve CoT faithfulness in general, with promising applications for CoT monitoring. Code for experiments in this paper is available at https://github.com/peterbhase/counterfactual-simulation-training