Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Small language models (SLMs) offer promising and efficient alternatives to large language models (LLMs). However, SLMs' limited capacity restricts their reasoning capabilities and makes them sensitive to prompt variations. To address these challenges, we propose a novel framework that enhances SLM reasoning capabilities through LLM generated blueprints. The blueprints provide structured, high-level reasoning guides that help SLMs systematically tackle related problems. Furthermore, our framework integrates a prompt template search mechanism to mitigate the SLMs' sensitivity to prompt variations. Our framework demonstrates improved SLM performance across various tasks, including math (GSM8K), coding (MBPP), and logic reasoning (BBH). Our approach improves the reasoning capabilities of SLMs without increasing model size or requiring additional training, offering a lightweight and deployment-friendly solution for on-device or resource-constrained environments.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 21 of its 29 references that are in the map. First posted on arXiv 10 Jun 2025.
Builds on
21 of 29 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Mistral 7Bcited by 3,164
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
- 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
8 more
- 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
- 2023Larger language models do in-context learning differentlycited by 394
- 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
- 2022Teaching Small Language Models to Reasoncited by 284
- 2023Large Language Model Guided Tree-of-Thoughtcited by 221
- 2023Efficient Large Language Models: A Surveycited by 186
- 2023Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialoguecited by 119
- 2024Are Large Language Models Good Prompt Optimizers?cited by 37
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Guiding Reasoning in Small Language Models with LLM Assistance
- 2025Towards Reasoning Ability of Small Language Models
- 2025Slm-mux: Orchestrating small language models for reasoning
- 2025SLR: Automated Synthesis for Scalable Logical Reasoning
- 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
- 2024Small Language Models: Survey, Measurements, and Insights
- 2026Offline Guidance, Online Reasoning: Reusing LLM Feedback for Small Language Models
- 2026Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs