# START: Self-taught Reasoner with Tools

arXiv: https://arxiv.org/abs/2503.04625 · HTML: https://mlsanity.com/paper/2503.04625/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 10 of 10)

- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Scaling Relationship on Learning Mathematical Reasoning with Large Language Models](https://mlsanity.com/paper/2308.01825/) (2023)
- [InternLM2 Technical Report](https://mlsanity.com/paper/2403.17297/) (2024)
- [O1 Replication Journey: A Strategic Progress Report -- Part 1](https://mlsanity.com/paper/2410.18982/) (2024)

## Built on by (most cited first, 30 of 47)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?](https://mlsanity.com/paper/2503.24235/) (2025)
- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [Deep Research Agents: A Systematic Examination And Roadmap](https://mlsanity.com/paper/2506.18096/) (2025)
- [Memento: Fine-tuning LLM Agents without Fine-tuning LLMs](https://mlsanity.com/paper/2508.16153/) (2025)
- [Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning](https://mlsanity.com/paper/2508.04416/) (2025)
- [Privileged Information Distillation for Language Models](https://mlsanity.com/paper/2602.04942/) (2026)
- [Distilling LLM Agent into Small Models with Retrieval and Code Tools](https://mlsanity.com/paper/2505.17612/) (2025)
- [Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling](https://mlsanity.com/paper/2505.11792/) (2025)
- [AutoPDL: Automatic Prompt Optimization for LLM Agents](https://mlsanity.com/paper/2504.04365/) (2025)
- [TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics](https://mlsanity.com/paper/2510.07181/) (2025)
- [MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability](https://mlsanity.com/paper/2505.20285/) (2025)
- [Learning to Reason via Mixture-of-Thought for Logical Reasoning](https://mlsanity.com/paper/2505.15817/) (2025)
- [Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning](https://mlsanity.com/paper/2509.23285/) (2025)
- [AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent](https://mlsanity.com/paper/2512.20745/) (2025)
- [Reasoning Beyond Limits: Advances and Open Problems for LLMs](https://mlsanity.com/paper/2503.22732/) (2025)
- [Learning from Peers in Reasoning Models](https://mlsanity.com/paper/2505.07787/) (2025)
- [CoRT: Code-integrated Reasoning within Thinking](https://mlsanity.com/paper/2506.09820/) (2025)
- [T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models](https://mlsanity.com/paper/2504.04718/) (2025)
- [a1: Steep Test-time Scaling Law via Environment Augmented Generation](https://mlsanity.com/paper/2504.14597/) (2025)
- [The Art of Scaling Test-Time Compute for Large Language Models](https://mlsanity.com/paper/2512.02008/) (2025)
- [InfoFlow: Reinforcing Search Agent Via Reward Density Optimization](https://mlsanity.com/paper/2510.26575/) (2025)
- [Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task](https://mlsanity.com/paper/2512.10359/) (2025)
- [THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning](https://mlsanity.com/paper/2509.13761/) (2025)
- [Trust but Verify! A Survey on Verification Design for Test-time Scaling](https://mlsanity.com/paper/2508.16665/) (2025)
- [Achieving Olympiad-Level Geometry Large Language Model Agent via Complexity Boosting Reinforcement Learning](https://mlsanity.com/paper/2512.10534/) (2025)
- [ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoning](https://mlsanity.com/paper/2510.07768/) (2025)
- [Teaching Language Models to Reason with Tools](https://mlsanity.com/paper/2510.20342/) (2025)
- [Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism](https://mlsanity.com/paper/2510.13170/) (2025)
- [ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration](https://mlsanity.com/paper/2601.06860/) (2026)

## Abstract (arXiv)

Large reasoning models (LRMs) like OpenAI-o1 and DeepSeek-R1 have demonstrated remarkable capabilities in complex reasoning tasks through the utilization of long Chain-of-thought (CoT). However, these models often suffer from hallucinations and inefficiencies due to their reliance solely on internal reasoning processes. In this paper, we introduce START (Self-Taught Reasoner with Tools), a novel tool-integrated long CoT reasoning LLM that significantly enhances reasoning capabilities by leveraging external tools. Through code execution, START is capable of performing complex computations, self-checking, exploring diverse methods, and self-debugging, thereby addressing the limitations of LRMs. The core innovation of START lies in its self-learning framework, which comprises two key techniques: 1) Hint-infer: We demonstrate that inserting artificially designed hints (e.g., ``Wait, maybe using Python here is a good idea.'') during the inference process of a LRM effectively stimulates its ability to utilize external tools without the need for any demonstration data. Hint-infer can also serve as a simple and effective sequential test-time scaling method; 2) Hint Rejection Sampling Fine-Tuning (Hint-RFT): Hint-RFT combines Hint-infer and RFT by scoring, filtering, and modifying the reasoning trajectories with tool invocation generated by a LRM via Hint-infer, followed by fine-tuning the LRM. Through this framework, we have fine-tuned the QwQ-32B model to achieve START. On PhD-level science QA (GPQA), competition-level math benchmarks (AMC23, AIME24, AIME25), and the competition-level code benchmark (LiveCodeBench), START achieves accuracy rates of 63.6%, 95.0%, 66.7%, 47.1%, and 47.3%, respectively. It significantly outperforms the base QwQ-32B and achieves performance comparable to the state-of-the-art open-weight model R1-Distill-Qwen-32B and the proprietary model o1-Preview.