mlsanityData to 14 Oct 2026

A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directions

arXiv 2504.09100 ↗First posted 12 Apr 2025cs.AI cs.CLcited by 4

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recently, the reasoning capabilities of large reasoning models (LRMs), such as DeepSeek-R1, have seen significant advancements through the slow thinking process. Despite these achievements, the substantial computational demands of LRMs present considerable challenges. In contrast, small reasoning models (SRMs), often distilled from larger ones, offer greater efficiency and can exhibit distinct capabilities and cognitive trajectories compared to LRMs. This work surveys around 170 recently published papers on SRMs for tackling various complex reasoning tasks. We review the current landscape of SRMs and analyze diverse training and inference techniques related to SRMs. Furthermore, we provide a comprehensive review of SRMs for domain-specific applications and discuss possible future research directions. This survey serves as an essential reference for researchers to leverage or develop SRMs for advanced reasoning functionalities with high efficiency.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 84 of its 166 references that are in the map. First posted on arXiv 12 Apr 2025.

Builds on

84 of 166 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond13 Mar
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models16 Jan
  3. 2024Multi-Step Reasoning with Large Language Models, a Survey16 Jul
  4. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning20 Jun
  5. 2023Code Llama: Open Foundation Models for Code24 Aug
  6. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 78

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  11. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  12. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
38 more
  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  2. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  3. 2025s1: Simple test-time scalingcited by 1,341
  4. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  7. 2023BloombergGPT: A Large Language Model for Financecited by 939
  8. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  9. 2024A Survey on Large Language Models for Code Generationcited by 658
  10. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  11. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  12. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  13. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  14. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  15. 2024InternLM2 Technical Reportcited by 397
  16. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  17. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 290
  18. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 290
  19. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  20. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  21. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  22. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  23. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  24. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 241
  25. 2024DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datacited by 234
  26. 2023Large Language Model Guided Tree-of-Thoughtcited by 221
  27. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 207
  28. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  29. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  30. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 148
  31. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  32. 2024Stream of Search (SoS): Learning to Search in Languagecited by 145
  33. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
  34. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 141
  35. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 122
  36. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 120
  37. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 119
  38. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoningcited by 110

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  2. 2025Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignmentcited by 7
  3. 2025HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generationcited by 2
  4. 2026Benchmarking Small Language Models and Small Reasoning Language Models on System Log Severity Classificationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Effective Learning for Small Reasoning Models: An Empirical Study on 0.5B Reasoning LLMs
  2. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
  3. 2025Efficient Inference for Large Reasoning Models: A Survey
  4. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
  5. 2025Logical Reasoning in Large Language Models: A Survey
  6. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models
  7. 2025Towards Reasoning Ability of Small Language Models
  8. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models