# Effectiveness of Chain-of-Thought in Distilling Reasoning Capability from Large Language Models

arXiv: https://arxiv.org/abs/2511.05184 · HTML: https://mlsanity.com/paper/2511.05184/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 7 of its 37 references that are in the map.
- First posted on arXiv 7 Nov 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 7 of 7)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [Small Language Models: Survey, Measurements, and Insights](https://mlsanity.com/paper/2409.15790/) (2024)
- [LLM Pruning and Distillation in Practice: The Minitron Approach](https://mlsanity.com/paper/2408.11796/) (2024)
- [Knowledge Distillation of Black-Box Large Language Models](https://mlsanity.com/paper/2401.07013/) (2024)

## Built on by (most cited first, 4 of 4)

- [The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systems](https://mlsanity.com/paper/2602.05182/) (2026)
- [Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch](https://mlsanity.com/paper/2603.22056/) (2026)
- [ExecTune: Effective Steering of Black-Box LLMs with Guide Models](https://mlsanity.com/paper/2604.09741/) (2026)
- [Structural Rationale Distillation via Reasoning Space Compression](https://mlsanity.com/paper/2605.07139/) (2026)

## Abstract (arXiv)

Chain-of-Thought (CoT) prompting is a widely used method to improve the reasoning capability of Large Language Models (LLMs). More recently, CoT has been leveraged in Knowledge Distillation (KD) to transfer reasoning capability from a larger LLM to a smaller one. This paper examines the role of CoT in distilling the reasoning capability from larger LLMs to smaller LLMs using white-box KD, analysing its effectiveness in improving the performance of the distilled models for various natural language reasoning and understanding tasks. We conduct white-box KD experiments using LLMs from the Qwen and Llama2 families, employing CoT data from the CoT-Collection dataset. The distilled models are then evaluated on natural language reasoning and understanding tasks from the BIG-Bench-Hard (BBH) benchmark, which presents complex challenges for smaller LLMs. Experimental results demonstrate the role of CoT in improving white-box KD effectiveness, enabling the distilled models to achieve better average performance in natural language reasoning and understanding tasks from BBH.