mlsanityData to 7 Oct 2026

Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation

arXiv 2503.16385 ↗First posted 20 Mar 2025cs.AIcited by 39

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Recent advancements in large language models (LLMs) have demonstrated remarkable reasoning capabilities through long chain-of-thought (CoT) reasoning. The R1 distillation scheme has emerged as a promising approach for training cost-effective models with enhanced reasoning abilities. However, the underlying mechanisms driving its effectiveness remain unclear. This study examines the universality of distillation data and identifies key components that enable the efficient transfer of long-chain reasoning capabilities in LLM distillation. Our findings reveal that the effectiveness of long CoT reasoning distillation from teacher models like Qwen-QwQ degrades significantly on nonhomologous models, challenging the assumed universality of current distillation methods. To gain deeper insights into the structure and patterns of long CoT reasoning, we propose DLCoT (Deconstructing Long Chain-of-Thought), a distillation data enhancement framework. DLCoT consists of three key steps: (1) data segmentation to decompose complex long CoT structures, (2) simplification by eliminating unsolvable and redundant solutions, and (3) optimization of intermediate error states. Our approach significantly improves model performance and token efficiency, facilitating the development of high-performance LLMs.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 22 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

13 of 22 references.

References · 13

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  8. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  9. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  10. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 140
  11. 2024O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?cited by 90
  12. 2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
1 more
  1. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 24; 2026: 15.

39 papers.

Sort:

By area:

  1. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  2. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  3. 2025Efficient Reasoning Models: A Surveycited by 93
  4. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 23
  5. 2026Purified OPSD: On-Policy Self-Distillation Without Losing How to Thinkcited by 23
  6. 2025Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoningcited by 19
  7. 2025Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systemscited by 13
  8. 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Promptingcited by 13
  9. 2025Test-time Prompt Interventioncited by 12
  10. 2025Learning from Diverse Reasoning Paths with Routing and Collaborationcited by 12
  11. 2025Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoningcited by 11
  12. 2025Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoningcited by 11
27 more
  1. 2025Incentivizing Strong Reasoning from Weak Supervisioncited by 8
  2. 2025LayoutCoT: Unleashing the Deep Reasoning Potential of Large Language Models for Layout Generationcited by 6
  3. 2025Merge-of-Thought Distillationcited by 6
  4. 2025Path Drift in Large Reasoning Models:How First-Person Commitments Override Safetycited by 5
  5. 2025CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoningcited by 5
  6. 2026Chain Of Thought Compression: A Theoretical Analysiscited by 5
  7. 2026Efficient Reasoning with Balanced Thinkingcited by 4
  8. 2025Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalizationcited by 3
  9. 2026Training-Trajectory-Aware Token Selectioncited by 3
  10. 2025Beyond Templates: Dynamic Adaptation of Reasoning Demonstrations via Feasibility-Aware Explorationcited by 2
  11. 2025The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Modelscited by 2
  12. 2025Investigating Thinking Behaviours of Reasoning-Based Language Models for Social Bias Mitigationcited by 2
  13. 2026Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoningcited by 2
  14. 2026Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Modelscited by 2
  15. 2025Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFNcited by 1
  16. 2025TRUST: A Decentralized Framework for Auditing Large Language Model Reasoningcited by 1
  17. 2025Understanding Chain-of-Thought in Large Language Models via Topological Data Analysiscited by 1
  18. 2026EntroCoT: Enhancing Chain-of-Thought via Adaptive Entropy-Guided Segmentationcited by 1
  19. 2026Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detectioncited by 1
  20. 2026DRScaffold: Boosting Dense-Scene Reasoning in Lightweight Vision Language Modelscited by 1
  21. 2025Balanced Actor Initialization: Stable RLHF Training of Distillation-Based Reasoning Modelscited by 0
  22. 2026ENTRA: Entropy-Based Redundancy Avoidance in Large Language Model Reasoningcited by 0
  23. 2026D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Modelscited by 0
  24. 2026TRACES: Tagging Reasoning Steps for Adaptive Cost-Efficient Early-Stoppingcited by 0
  25. 2026Generative Spatiotemporal Intent Sequence Recommendation via Implicit Reasoning in Amapcited by 0
  26. 2026Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Tracescited by 0
  27. 2026Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
  2. 2025Efficient Long CoT Reasoning in Small Language Models
  3. 2022Distilling Reasoning Capabilities into Smaller Language Models
  4. 2025Effectiveness of Chain-of-Thought in Distilling Reasoning Capability from Large Language Models
  5. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
  6. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
  7. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  8. 2025Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models