mlsanityData to 14 Oct 2026

DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models

arXiv 2505.13975 ↗First posted 20 May 2025cs.CLcited by 30

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

While Large Reasoning Models (LRMs) have demonstrated success in complex reasoning tasks through long chain-of-thought (CoT) reasoning, their inference often involves excessively verbose reasoning traces, resulting in substantial inefficiency. To address this, we propose Distilled Reasoning Pruning (DRP), a hybrid framework that combines inference-time pruning with tuning-based distillation, two widely used strategies for efficient reasoning. DRP uses a teacher model to perform skill-aware step decomposition and content pruning, and then distills the pruned reasoning paths into a student model, enabling it to reason both efficiently and accurately. Across several challenging mathematical reasoning datasets, we find that models trained with DRP achieve substantial improvements in token efficiency without sacrificing accuracy. Specifically, DRP reduces average token usage on GSM8K from 917 to 328 while improving accuracy from 91.7% to 94.1%, and achieves a 43% token reduction on AIME with no performance drop. Further analysis shows that aligning the reasoning structure of training CoTs with the student's reasoning capacity is critical for effective knowledge transfer and performance gains.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 38 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

20 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance31 Mar
  2. 2025Small Models Struggle to Learn from Strong Reasoners17 Feb

Other references · 18

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2025s1: Simple test-time scalingcited by 1,341
  6. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  7. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  8. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  9. 2025Dynamic Early Exit in Reasoning Modelscited by 219
  10. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 205
  11. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 169
  12. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
6 more
  1. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 125
  2. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  3. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 70
  4. 2024Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoningcited by 69
  5. 2025Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspectivecited by 29
  6. 2024Stronger Models are NOT Stronger Teachers for Instruction Tuningcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 22.

30 papers.

Sort:

By area:

  1. 2025Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropycited by 38
  2. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  3. 2026SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Modelscited by 21
  4. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  5. 2025Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoningcited by 17
  6. 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystemscited by 17
  7. 2025TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning Compressioncited by 16
  8. 2025Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoningcited by 16
  9. 2025Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoningcited by 15
  10. 2026ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agentscited by 14
  11. 2026Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-offcited by 13
  12. 2026Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidancecited by 13
18 more
  1. 2026Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillationcited by 11
  2. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 8
  3. 2026Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertaintycited by 6
  4. 2026FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoningcited by 5
  5. 2026ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressurecited by 4
  6. 2026DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderationcited by 4
  7. 2026MIS-HCC: Hierarchical Channel Clustering for Efficient Medical Image Segmentationcited by 4
  8. 2026MUSE: Resolving Manifold Misalignment in Visual Tokenization via Topological Orthogonalitycited by 3
  9. 2026Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Accesscited by 3
  10. 2025Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoningcited by 2
  11. 2026Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penaltycited by 2
  12. 2026TRiMS: Real-Time Tracking of Minimal Sufficient Length for Efficient Reasoning via RLcited by 2
  13. 2026Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elitecited by 1
  14. 2026Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMscited by 0
  15. 2026MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignmentcited by 0
  16. 2026Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoningcited by 0
  17. 2026DRT: Dense Reasoning Trace for Efficient and Grounded Multimodal Reasoningcited by 0
  18. 2026Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
  2. 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting
  3. 2025Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
  4. 2025Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning
  5. 2025Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models
  6. 2025Think Clearly: Improving Reasoning via Redundant Token Pruning
  7. 2026Skill-Aware Data Selection and Fine-Tuning for Data-Efficient Reasoning Distillation
  8. 2025Not All Correct Answers Are Equal: Why Your Distillation Source Matters