# Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness

arXiv: https://arxiv.org/abs/2405.18915 · HTML: https://mlsanity.com/paper/2405.18915/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 9 of its 48 references that are in the map.
- First posted on arXiv 29 May 2024.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2402.13950/) (2024)
- [On Measuring Faithfulness or Self-consistency of Natural Language Explanations](https://mlsanity.com/paper/2311.07466/) (2023)
- [From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning](https://mlsanity.com/paper/2310.00492/) (2023)
- [How Likely Do LLMs with CoT Mimic Human Reasoning?](https://mlsanity.com/paper/2402.16048/) (2024)
- [Gradient based Feature Attribution in Explainable AI: A Technical Review](https://mlsanity.com/paper/2403.10415/) (2024)

## Built on by (most cited first, 20 of 20)

- [Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!](https://mlsanity.com/paper/2504.09762/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models](https://mlsanity.com/paper/2409.19667/) (2024)
- [CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning](https://mlsanity.com/paper/2410.10336/) (2024)
- [GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaboration](https://mlsanity.com/paper/2410.18032/) (2024)
- [A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models](https://mlsanity.com/paper/2505.23945/) (2025)
- [Do Cognitively Interpretable Reasoning Traces Improve LLM Performance?](https://mlsanity.com/paper/2508.16695/) (2025)
- [Unveiling Confirmation Bias in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.12301/) (2025)
- [Interpretable Traces, Unexpected Outcomes: Investigating the Disconnect in Trace-Based Knowledge Distillation](https://mlsanity.com/paper/2505.13792/) (2025)
- [Thoughts without Thinking: Reconsidering the Explanatory Value of Chain-of-Thought Reasoning in LLMs through Agentic Pipelines](https://mlsanity.com/paper/2505.00875/) (2025)
- [SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control](https://mlsanity.com/paper/2508.20018/) (2025)
- [See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection](https://mlsanity.com/paper/2604.24339/) (2026)
- [Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy](https://mlsanity.com/paper/2605.25603/) (2026)
- [ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails](https://mlsanity.com/paper/2605.31073/) (2026)
- [Reasoning About Reasoning: Towards Informed and Reflective Use of LLM Reasoning in HCI](https://mlsanity.com/paper/2510.22978/) (2025)
- [Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models](https://mlsanity.com/paper/2602.22072/) (2026)
- [Ambient Persuasion in a Deployed AI Agent: Unauthorized Escalation Following Routine Non-Adversarial Content Exposure](https://mlsanity.com/paper/2605.00055/) (2026)
- [Agentic Interpretation: Lattice-Structured Evidence for LLM-Based Program Analysis](https://mlsanity.com/paper/2605.12694/) (2026)
- [Context-Masked Truncated Reasoning Audits for Answer-Key Dependence in LLM Tutors](https://mlsanity.com/paper/2607.04572/) (2026)
- [A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models](https://mlsanity.com/paper/2607.26102/) (2026)

## Abstract (arXiv)

Chain-of-thought (CoT) prompting demonstrates varying performance under different reasoning tasks. Previous work attempts to evaluate it but falls short in providing an in-depth analysis of patterns that influence the CoT. In this paper, we study the CoT performance from the perspective of effectiveness and faithfulness. For the former, we identify key factors that influence CoT effectiveness on performance improvement, including problem difficulty, information gain, and information flow. For the latter, we interpret the unfaithful CoT issue by conducting a joint analysis of the information interaction among the question, CoT, and answer. The result demonstrates that, when the LLM predicts answers, it can recall correct information missing in the CoT from the question, leading to the problem. Finally, we propose a novel algorithm to mitigate this issue, in which we recall extra information from the question to enhance the CoT generation and evaluate CoTs based on their information gain. Extensive experiments demonstrate that our approach enhances both the faithfulness and effectiveness of CoT.