mlsanityData to 14 Oct 2026

Towards Efficient CoT Distillation: Self-Guided Rationale Selector for Better Performance with Fewer Rationales

arXiv 2509.23574 ↗First posted 28 Sep 2025cs.CL cs.AIcited by 8

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) distillation aims to enhance small language models' (SLMs) reasoning by transferring multi-step reasoning capability from the larger teacher models. However, existing work underestimates rationale quality, focusing primarily on data quantity, which may transfer noisy or incorrect information to the student model. To address the above issues, we proposed Model-Oriented Rationale Selection Distillation (MoRSD), which can discern and select high quality rationales for distillation to improve performance further. We further propose a Rationale Difficulty (RD) metric to measure the ability of the student model to generate the correct answer under a given rationale. Compared to the baseline, we achieved 4.6% average improvement on seven datasets over three tasks, using fewer rationales by controlling their accuracy, diversity, and difficulty. Our results reveal that a small portion of the high quality rationales can enhance the reasoning ability of student models than the entire dataset. Our method promises to be a possible solution for efficient CoT distillation. Our code will be released in https://github.com/Leon221220/MoRSD.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 39 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

28 of 39 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Distillation Scaling Laws12 Feb
  2. 2024Mentor-KD: Making Small Language Models Better Multi-step Reasoners11 Oct
  3. 2024Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation30 May
  4. 2023MCC-KD: Multi-CoT Consistent Knowledge Distillation23 Oct
  5. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step24 Jun
  6. 2023SCOTT: Self-Consistent Chain-of-Thought Distillation3 May
  7. 2022Large Language Models Are Reasoning Teachers20 Dec
  8. 2022Large Language Models are Zero-Shot Reasoners24 May

Other references · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2023LIMA: Less Is More for Alignmentcited by 1,100
  7. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  8. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  9. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  10. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  11. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  12. 2016Solving General Arithmetic Word Problemscited by 360
8 more
  1. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 260
  2. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 251
  3. 2022PINTO: Faithful Language Reasoning Using Prompt-Generated Rationalescited by 56
  4. 2024Interactive AI with Retrieval-Augmented Generation for Next Generation Networkingcited by 34
  5. 2024Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Modelscited by 32
  6. 2024Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planningcited by 13
  7. 2024ELAD: Explanation-Guided Large Language Models Active Distillationcited by 12
  8. 2024LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvementcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningcited by 17
  2. 2026Student-in-the-Loop Chain-of-Thought Distillation via Generation-Time Selectioncited by 4
  3. 2026From Fact Overwriting to Knowledge Evolution: Causal Editing via On-Policy Self-Distillationcited by 1
  4. 2026Structural Rationale Distillation via Reasoning Space Compressioncited by 0
  5. 2026Characterize Then Distill: Mechanistic Reasoning in Large Output Spacescited by 0
  6. 2026Decision Shifts, Lost Label Functionality, and an Inconclusive Grounding Audit in Correctness-Gated Multi-Teacher Distillationcited by 0
  7. 2026Overtaught, Underlearned: Learner-Centered Distillation from Multi-Agent Discussioncited by 0
  8. 2026Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Investigating Mysteries of CoT-Augmented Distillation
  2. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
  3. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
  4. 2022Distilling Reasoning Capabilities into Smaller Language Models
  5. 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
  6. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
  7. 2026Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
  8. 2025Effectiveness of Chain-of-Thought in Distilling Reasoning Capability from Large Language Models