mlsanityData to 14 Oct 2026

Small Models Struggle to Learn from Strong Reasoners

arXiv 2502.12143 ↗First posted 17 Feb 2025cs.AIcited by 113

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) excel in complex reasoning tasks, and distilling their reasoning capabilities into smaller models has shown promise. However, we uncover an interesting phenomenon, which we term the Small Model Learnability Gap: small models (≤3B parameters) do not consistently benefit from long chain-of-thought (CoT) reasoning or distillation from larger models. Instead, they perform better when fine-tuned on shorter, simpler reasoning chains that better align with their intrinsic learning capacity. To address this, we propose Mix Distillation, a simple yet effective strategy that balances reasoning complexity by combining long and short CoT examples or reasoning from both larger and smaller models. Our experiments demonstrate that Mix Distillation significantly improves small model reasoning performance compared to training on either data alone. These findings highlight the limitations of direct strong model distillation and underscore the importance of adapting reasoning complexity for effective reasoning capability transfer.

Key facts. Cited by 113 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 47 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

36 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation21 Feb

Other references · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2020Knowledge Distillation: A Surveycited by 1,603
  9. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  10. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  11. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  12. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
23 more
  1. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  2. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  3. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  4. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  5. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  6. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  7. 2019On the Efficacy of Knowledge Distillationcited by 381
  8. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  9. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  10. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  11. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 257
  12. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 200
  13. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 164
  14. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 155
  15. 2024MAmmoTH2: Scaling Instructions from the Webcited by 145
  16. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scalingcited by 77
  17. 2025The Best Instruction-Tuning Data are Those That Fitcited by 52
  18. 2024Evaluating Language Models as Synthetic Data Generatorscited by 33
  19. 2023Lifting the Curse of Capacity Gap in Distilling Language Modelscited by 25
  20. 2024PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuningcited by 25
  21. 2024Stronger Models are NOT Stronger Teachers for Instruction Tuningcited by 16
  22. 2024Demystifying μcited by 7
  23. 2024Smaller Language Models Are Better Instruction Evolverscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 54; 2026: 59.

113 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  2. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 299
  3. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  4. 2025Thinkless: LLM Learns When to Thinkcited by 113
  5. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 106
  6. 2025Efficient Reasoning Models: A Surveycited by 95
  7. 2025BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoningcited by 63
  8. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 62
  9. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Mergingcited by 58
  10. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
  11. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  12. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
38 more
  1. 2025Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learningcited by 38
  2. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  3. 2025DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Modelscited by 30
  4. 2025Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Propertiescited by 29
  5. 2026Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoningcited by 27
  6. 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Whycited by 25
  7. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  8. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 19
  9. 2025Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoningcited by 19
  10. 2026Demystifying On-Policy Distillation: Roles, Pathologies, and Regulationscited by 19
  11. 2025LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perceptioncited by 16
  12. 2025When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasonerscited by 16
  13. 2026X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMscited by 16
  14. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  15. 2025Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillationcited by 15
  16. 2025Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcementcited by 14
  17. 2025Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agentscited by 14
  18. 2025Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoningcited by 14
  19. 2025Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Modelscited by 13
  20. 2026Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignmentcited by 13
  21. 2026DOPD: Dual On-policy Distillationcited by 12
  22. 2025AutoSDT: Scaling Data-Driven Discovery Tasks Toward Open Co-Scientistscited by 11
  23. 2025Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoningcited by 11
  24. 2025Reinforcement Learning Teachers of Test Time Scalingcited by 10
  25. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillationcited by 9
  26. 2025MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistantscited by 9
  27. 2026Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertaintycited by 9
  28. 2025The Signal is in the Steps: Local Scoring for Reasoning Data Selectioncited by 8
  29. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 8
  30. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 7
  31. 2026On-Policy Distillation with Best-of-N Teacher Rollout Selectioncited by 7
  32. 2025RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Samplingcited by 6
  33. 2025LoRA is All You Need for Safety Alignment of Reasoning LLMscited by 6
  34. 2025ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoningcited by 6
  35. 2026Long-Chain Reasoning Distillation via Adaptive Prefix Alignmentcited by 6
  36. 2025Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theorycited by 5
  37. 2025In Good GRACEs: Principled Teacher Selection for Knowledge Distillationcited by 5
  38. 2026Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVRcited by 5

50 most cited of 113 All 113 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Mixed Distillation Helps Smaller Language Model Better Reasoning
  2. 2026Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
  3. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
  4. 2022Distilling Reasoning Capabilities into Smaller Language Models
  5. 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
  6. 2025Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
  7. 2025Towards Understanding Distilled Reasoning Models: A Representational Approach
  8. 2024Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation