# Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math

arXiv: https://arxiv.org/abs/2504.21233 · HTML: https://mlsanity.com/paper/2504.21233/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning](https://mlsanity.com/paper/2309.05653/) (2023)
- [From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models](https://mlsanity.com/paper/2406.16838/) (2024)
- [SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM](https://mlsanity.com/paper/2504.14286/) (2025)
- [Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problems](https://mlsanity.com/paper/2009.11506/) (2020)
- [On the Emergence of Thinking in LLMs I: Searching for the Right Intuition](https://mlsanity.com/paper/2502.06773/) (2025)

## Built on by (most cited first, 30 of 44)

- [Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers](https://mlsanity.com/paper/2506.15674/) (2025)
- [VeriThinker: Learning to Verify Makes Reasoning Model Efficient](https://mlsanity.com/paper/2505.17941/) (2025)
- [BLEUBERI: BLEU is a surprisingly effective reward for instruction following](https://mlsanity.com/paper/2505.11080/) (2025)
- [Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.09853/) (2025)
- [Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning](https://mlsanity.com/paper/2505.14216/) (2025)
- [Energy Use of AI Inference, Efficiency Pathways, and Test-Time Scaling](https://mlsanity.com/paper/2509.20241/) (2025)
- [Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training](https://mlsanity.com/paper/2510.20956/) (2025)
- [Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation](https://mlsanity.com/paper/2507.06607/) (2025)
- [OPRD: On-Policy Representation Distillation](https://mlsanity.com/paper/2606.06021/) (2026)
- [G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance](https://mlsanity.com/paper/2508.13023/) (2025)
- [Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility](https://mlsanity.com/paper/2608.04001/) (2026)
- [SHARP: Synthesizing High-quality Aligned Reasoning Problems for Large Reasoning Models Reinforcement Learning](https://mlsanity.com/paper/2505.14147/) (2025)
- [Path Drift in Large Reasoning Models:How First-Person Commitments Override Safety](https://mlsanity.com/paper/2510.10013/) (2025)
- [One Token Embedding Is Enough to Deadlock Your Large Reasoning Model](https://mlsanity.com/paper/2510.15965/) (2025)
- [Reasoning Up the Instruction Ladder for Controllable Language Models](https://mlsanity.com/paper/2511.04694/) (2025)
- [Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings](https://mlsanity.com/paper/2505.13718/) (2025)
- [Recall-Extend Dynamics: Enhancing Small Language Models through Controlled Exploration and Refined Offline Integration](https://mlsanity.com/paper/2508.16677/) (2025)
- [Poivre: Self-Refining Visual Pointing with Reinforcement Learning](https://mlsanity.com/paper/2509.23746/) (2025)
- [Learning to Reason as Action Abstractions with Scalable Mid-Training RL](https://mlsanity.com/paper/2509.25810/) (2025)
- [Concise Reasoning in the Lens of Lagrangian Optimization](https://mlsanity.com/paper/2510.10168/) (2025)
- [When Reasoning Meets Its Laws](https://mlsanity.com/paper/2512.17901/) (2025)
- [DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels](https://mlsanity.com/paper/2602.11715/) (2026)
- [Learning from Partial Chain-of-Thought via Truncated-Reasoning Self-Distillation](https://mlsanity.com/paper/2603.13274/) (2026)
- [In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners](https://mlsanity.com/paper/2509.22230/) (2025)
- [LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models](https://mlsanity.com/paper/2510.11031/) (2025)
- [Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense](https://mlsanity.com/paper/2510.16259/) (2025)
- [UrduBench: An Urdu Reasoning Benchmark using Contextually Ensembled Translations with Human-in-the-Loop](https://mlsanity.com/paper/2601.21000/) (2026)
- [HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models](https://mlsanity.com/paper/2604.12229/) (2026)
- [Think Multilingual, Not Harder: A Data-Efficient Framework for Teaching Reasoning Models to Code-Switch](https://mlsanity.com/paper/2604.15490/) (2026)
- [Sliding-window beats linear attention](https://mlsanity.com/paper/2608.28444/) (2026)

## Abstract (arXiv)

Chain-of-Thought (CoT) significantly enhances formal reasoning capabilities in Large Language Models (LLMs) by training them to explicitly generate intermediate reasoning steps. While LLMs readily benefit from such techniques, improving reasoning in Small Language Models (SLMs) remains challenging due to their limited model capacity. Recent work by Deepseek-R1 demonstrates that distillation from LLM-generated synthetic data can substantially improve the reasoning ability of SLM. However, the detailed modeling recipe is not disclosed. In this work, we present a systematic training recipe for SLMs that consists of four steps: (1) large-scale mid-training on diverse distilled long-CoT data, (2) supervised fine-tuning on high-quality long-CoT data, (3) Rollout DPO leveraging a carefully curated preference dataset, and (4) Reinforcement Learning (RL) with Verifiable Reward. We apply our method on Phi-4-Mini, a compact 3.8B-parameter model. The resulting Phi-4-Mini-Reasoning model exceeds, on math reasoning tasks, much larger reasoning models, e.g., outperforming DeepSeek-R1-Distill-Qwen-7B by 3.2 points and DeepSeek-R1-Distill-Llama-8B by 7.7 points on Math-500. Our results validate that a carefully designed training recipe, with large-scale high-quality CoT data, is effective to unlock strong reasoning capabilities even in resource-constrained small models.