mlsanityData to 14 Oct 2026

Tailoring the Curriculum: Student-Centered Reasoning Distillation via Dynamic Data-Model Compatibility

arXiv 2605.29229 ↗First posted 28 May 2026cs.AIcited by 2

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Reasoning distillation transfers complex reasoning abilities from large language models (LLMs) to smaller ones, yet its success depends on how well the training data align with the student model. This paper introduces the Data-Model Compatibility (DMC) metric, which can be used to assess the suitability of a dataset for reasoning distillation on a student model. DMC provides an assessment by jointly considering data quality, relative difficulty, and student capability. We validated the effectiveness of DMC from two perspectives: (1) DMC exhibits a strong correlation with reasoning distillation performance; and (2) using DMC as the criterion for data selection leads to improved reasoning distillation performance. Both findings are consistently demonstrated across multiple student models and tasks. Moreover, since the DMC of each dataset dynamically changes during training, our experiments demonstrate that dynamically selecting datasets based on DMC can further enhance performance.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 44 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

29 of 44 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation24 May
  2. 2025Not All Correct Answers Are Equal: Why Your Distillation Source Matters20 May
  3. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning25 Feb
  4. 2024Mentor-KD: Making Small Language Models Better Multi-step Reasoners11 Oct

Other references · 25

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2023Mistral 7Bcited by 3,164
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  11. 2024OpenAI o1 System Cardcited by 1,244
  12. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
13 more
  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  2. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  3. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  4. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 327
  5. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 214
  6. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  7. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
  8. 2024GenQA: Generating Millions of Instructions from a Handful of Promptscited by 35
  9. 2024SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflectioncited by 34
  10. 2024Reverse Thinking Makes LLMs Stronger Reasonerscited by 29
  11. 2025Dancing with Critiques: Enhancing LLM Reasoning with Stepwise Natural Language Self-Critiquecited by 13
  12. 2025ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflectioncited by 12
  13. 2025CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdomcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvementcited by 0
  2. 2026Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
  2. 2026Collaborative Reasoning Distillation via Cross-Feedback and Coherent Curation
  3. 2026Curriculum Learning-Guided Progressive Distillation in Large Language Models
  4. 2026Validity-Calibrated Reasoning Distillation
  5. 2026Skill-Aware Data Selection and Fine-Tuning for Data-Efficient Reasoning Distillation
  6. 2025Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation
  7. 2025Small Models Struggle to Learn from Strong Reasoners
  8. 2026Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information