mlsanityData to 14 Oct 2026

Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation

arXiv 2507.00054 ↗First posted 25 Jun 2025cs.AI cs.CL cs.LGcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

The push to compress and impart the proficiency of Large Language Models (LLMs) into more deployable and efficient Small Language Models (SLMs) has benefited from improvements in knowledge distillation (KD) techniques. These techniques allow a smaller student model to learn from a more capable and larger teacher model's responses. However, distillation often revolves around the student model merely copying the teacher's in-distribution responses, limiting its generalisability. This limitation is amplified on reasoning tasks and can be computationally expensive. In this study, we propose AdvDistill, a reward-guided dataset distillation framework. We utilise multiple generations (responses) from a teacher for each prompt and assign rewards based on rule-based verifiers. These varying and normally distributed rewards serve as weights when training student models. Our methods and their subsequent behavioural analysis demonstrate a significant improvement in student model performance for mathematical and complex reasoning tasks, showcasing the efficacy and benefits of incorporating a rewarding mechanism in dataset distillation processes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 62 references that are in the map. First posted on arXiv 25 Jun 2025.

Builds on

25 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Towards Understanding Distilled Reasoning Models: A Representational Approach5 Mar

Other references · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  7. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  8. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
  9. 2024DistiLLM: Towards Streamlined Distillation for Large Language Modelscited by 163
  10. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solverscited by 158
  11. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 139
  12. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
12 more
  1. 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 90
  2. 2025DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMscited by 84
  3. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Samplingcited by 61
  4. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Timecited by 41
  5. 2024Progressive distillation induces an implicit curriculumcited by 15
  6. 2024Evolving Knowledge Distillation with Large Language Models and Active Learningcited by 12
  7. 2025Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?cited by 12
  8. 2025Who Taught You That? Tracing Teachers in Model Distillationcited by 8
  9. 2025On Teacher Hacking in Language Model Distillationcited by 6
  10. 2024CourseGPT-zh: an Educational Large Language Model Based on Knowledge Distillation Incorporating Prompt Optimizationcited by 5
  11. 2025Advantage-Guided Distillation for Preference Alignment in Small Language Modelscited by 5
  12. 2025Winning Big with Small Models: Knowledge Distillation vs. Self-Training for Reducing Hallucination in Product QA Agentscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation
  2. 2025Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
  3. 2026Curriculum Learning-Guided Progressive Distillation in Large Language Models
  4. 2024SIKeD: Self-guided Iterative Knowledge Distillation for mathematical reasoning
  5. 2025UNDO: Understanding Distillation as Optimization
  6. 2024Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
  7. 2026KDFP: A first-principles approach to knowledge distillation in large language models
  8. 2025Not All Correct Answers Are Equal: Why Your Distillation Source Matters