# PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving

arXiv: https://arxiv.org/abs/2507.07495 · HTML: https://mlsanity.com/paper/2507.07495/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning](https://mlsanity.com/paper/2212.07919/) (2022)
- [Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B](https://mlsanity.com/paper/2406.07394/) (2024)
- [NATURAL PLAN: Benchmarking LLMs on Natural Language Planning](https://mlsanity.com/paper/2406.04520/) (2024)
- [AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories](https://mlsanity.com/paper/2410.07706/) (2024)
- [Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents](https://mlsanity.com/paper/2402.11651/) (2024)
- [Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing](https://mlsanity.com/paper/2402.00658/) (2024)
- [Is a Question Decomposition Unit All We Need?](https://mlsanity.com/paper/2205.12538/) (2022)
- [Less is More: Summary of Long Instructions is Better for Program Synthesis](https://mlsanity.com/paper/2203.08597/) (2022)

## Built on by (most cited first, 6 of 6)

- [COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context](https://mlsanity.com/paper/2510.08790/) (2025)
- [ThinkTuning: Instilling Cognitive Reflections without Distillation](https://mlsanity.com/paper/2508.07616/) (2025)
- [CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs](https://mlsanity.com/paper/2509.00971/) (2025)
- [Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Model](https://mlsanity.com/paper/2603.24989/) (2026)
- [Decoupled Physical Modeling and Execution for Physics Reasoning](https://mlsanity.com/paper/2608.22126/) (2026)
- [PersonalPlan: Planning Multi-Agent Systems for Personalized Programming Learning](https://mlsanity.com/paper/2606.18633/) (2026)

## Abstract (arXiv)

Recently, decomposing complex problems into simple subtasks--a crucial part of human-like natural planning--to solve the given problem has significantly boosted the performance of large language models (LLMs). However, leveraging such planning structures during post-training to boost the performance of smaller open-source LLMs remains underexplored. Motivated by this, we introduce PLAN-TUNING, a unified post-training framework that (i) distills synthetic task decompositions (termed "planning trajectories") from large-scale LLMs and (ii) fine-tunes smaller models via supervised and reinforcement-learning objectives designed to mimic these planning processes to improve complex reasoning. On GSM8k and the MATH benchmarks, plan-tuned models outperform strong baselines by an average ~7%. Furthermore, plan-tuned models show better generalization capabilities on out-of-domain datasets, with average ~10% and ~12% performance improvements on OlympiadBench and AIME 2024, respectively. Our detailed analysis demonstrates how planning trajectories improves complex reasoning capabilities, showing that PLAN-TUNING is an effective strategy for improving task-specific performance of smaller LLMs.