# Thoughts without Thinking: Reconsidering the Explanatory Value of Chain-of-Thought Reasoning in LLMs through Agentic Pipelines

arXiv: https://arxiv.org/abs/2505.00875 · HTML: https://mlsanity.com/paper/2505.00875/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 1 May 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era](https://mlsanity.com/paper/2403.08946/) (2024)
- [Fully Autonomous AI Agents Should Not be Developed](https://mlsanity.com/paper/2502.02649/) (2025)
- [Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers](https://mlsanity.com/paper/2502.02393/) (2025)
- [Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness](https://mlsanity.com/paper/2405.18915/) (2024)
- [LLMs for XAI: Future Directions for Explaining Explanations](https://mlsanity.com/paper/2405.06064/) (2024)
- [Large Language Models Cannot Explain Themselves](https://mlsanity.com/paper/2405.04382/) (2024)
- [Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems: A Comprehensive Approach to Explainable Large Language Models](https://mlsanity.com/paper/2501.18645/) (2025)
- [Multi-Modal Data Exploration via Language Agents](https://mlsanity.com/paper/2412.18428/) (2024)
- [Addressing Social Misattributions of Large Language Models: An HCXAI-based Approach](https://mlsanity.com/paper/2403.17873/) (2024)
- [QA-TOOLBOX: Conversational Question-Answering for process task guidance in manufacturing](https://mlsanity.com/paper/2412.02638/) (2024)
- [Understanding Stakeholders' Perceptions and Needs Across the LLM Supply Chain](https://mlsanity.com/paper/2405.16311/) (2024)

## Built on by (most cited first, 7 of 7)

- [AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction](https://mlsanity.com/paper/2506.17784/) (2025)
- [Test-time Prompt Intervention](https://mlsanity.com/paper/2508.02511/) (2025)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations](https://mlsanity.com/paper/2511.12001/) (2025)
- [Vision-Language Models Mistake Head Orientation for Gaze Direction: Nonverbal Conversation Cues](https://mlsanity.com/paper/2506.05412/) (2025)
- [Interpreting Agentic Systems: Beyond Model Explanations to System-Level Accountability](https://mlsanity.com/paper/2601.17168/) (2026)
- [PCRLLM: Proof-Carrying Reasoning with Large Language Models under Stepwise Logical Constraints](https://mlsanity.com/paper/2511.08392/) (2025)

## Abstract (arXiv)

Agentic pipelines present novel challenges and opportunities for human-centered explainability. The HCXAI community is still grappling with how best to make the inner workings of LLMs transparent in actionable ways. Agentic pipelines consist of multiple LLMs working in cooperation with minimal human control. In this research paper, we present early findings from an agentic pipeline implementation of a perceptive task guidance system. Through quantitative and qualitative analysis, we analyze how Chain-of-Thought (CoT) reasoning, a common vehicle for explainability in LLMs, operates within agentic pipelines. We demonstrate that CoT reasoning alone does not lead to better outputs, nor does it offer explainability, as it tends to produce explanations without explainability, in that they do not improve the ability of end users to better understand systems or achieve their goals.