MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
While Large Language Models (LLMs) have emerged with remarkable capabilities in complex tasks through Chain-of-Thought reasoning, practical resource constraints have sparked interest in transferring these abilities to smaller models. However, achieving both domain performance and cross-domain generalization remains challenging. Existing approaches typically restrict students to following a single golden rationale and treat different reasoning paths independently. Due to distinct inductive biases and intrinsic preferences, alongside the student's evolving capacity and reasoning preferences during training, a teacher's "optimal" rationale could act as out-of-distribution noise. This misalignment leads to a degeneration of the student's latent reasoning distribution, causing suboptimal performance. To bridge this gap, we propose MIND, a capability-adaptive framework that transitions distillation from passive mimicry to active cognitive construction. We synthesize diverse teacher perspectives through a novel "Teaching Assistant" network. By employing a Feedback-Driven Inertia Calibration mechanism, this network utilizes inertia-filtered training loss to align supervision with the student's current adaptability, effectively enhancing performance while mitigating catastrophic forgetting. Extensive experiments demonstrate that MIND achieves state-of-the-art performance on both in-distribution and out-of-distribution benchmarks, and our sophisticated latent space analysis further confirms the mechanism of reasoning ability internalization.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 32 references that are in the map. First posted on arXiv 7 Jan 2026.
Builds on
9 of 32 references.
Closest ancestors · 3
Same direction, newest first
- 2024Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation30 May
- 2023MCC-KD: Multi-CoT Consistent Knowledge Distillation23 Oct
- 2023SCOTT: Self-Consistent Chain-of-Thought Distillation3 May
Other references · 6
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
- 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026"The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework
- 2026Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
- 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
- 2026H'an D=an Xu'e B`u (Mimicry) or Q=ing Ch=u Y'u L'an (Mastery)? A Cognitive Perspective on Reasoning Distillation in Large Language Models
- 2024Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation
- 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
- 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Models
- 2025SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought