mlsanityData to 7 Oct 2026

D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models

arXiv 2602.21786 ↗First posted 25 Feb 2026cs.CLcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) often induces "overthinking" in Small Language Models (SLMs), leading to performance degradation and excessive token consumption. In this study, we propose Disciplined Chain-of-Thought (D-CoT), a novel framework that enforces a structured reasoning process using control tags -- such as <TEMP_LOW> for fact-checking and <TEMP_HIGH> for multi-perspective exploration -- as auxiliary scaffolding during training. By optimizing the CoT trajectory, D-CoT suppresses reasoning drift and simultaneously achieves token reduction and performance improvement. We demonstrate the efficacy of our approach on Qwen3-8B: with only 5,000 training samples, D-CoT significantly boosts accuracy on GPQA-diamond by 9.9% and MMLU-Pro (0-shot) by 9.1%, while drastically reducing computational costs. Furthermore, we confirm that the model internalizes this disciplined thought structure, maintaining high performance even without explicit control tags during inference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 8 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

6 of 8 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation20 Mar
  2. 2025When More is Less: Understanding Chain-of-Thought Length in LLMs11 Feb

Other references · 4

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024AS-ES Learning: Towards Efficient CoT Learning in Small Models
  2. 2026S3-CoT: Self-Sampled Succinct Reasoning Enables Efficient Chain-of-Thought LLMs
  3. 2022Distilling Reasoning Capabilities into Smaller Language Models
  4. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  5. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
  6. 2025Effectiveness of Chain-of-Thought in Distilling Reasoning Capability from Large Language Models
  7. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
  8. 2025Skip-Thinking: Chunk-wise Chain-of-Thought Distillation Enable Smaller Language Models to Reason Better and Faster