# Explicit Planning Helps Language Models in Logical Reasoning

arXiv: https://arxiv.org/abs/2303.15714 · HTML: https://mlsanity.com/paper/2303.15714/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought](https://mlsanity.com/paper/2210.01240/) (2022)
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022)
- [Transformers as Soft Reasoners over Language](https://mlsanity.com/paper/2002.05867/) (2020)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [QASC: A Dataset for Question Answering via Sentence Composition](https://mlsanity.com/paper/1910.11473/) (2019)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics](https://mlsanity.com/paper/2112.08726/) (2021)
- [Language Model Cascades](https://mlsanity.com/paper/2207.10342/) (2022)
- [Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency](https://mlsanity.com/paper/1809.01812/) (2018)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [On the Paradox of Learning to Reason from Data](https://mlsanity.com/paper/2205.11502/) (2022)
- [Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2107.02794/) (2021)
- [Natural Language Deduction through Search over Statement Compositions](https://mlsanity.com/paper/2201.06028/) (2022)
- [Compositional Vector Space Models for Knowledge Base Completion](https://mlsanity.com/paper/1504.06662/) (2015)
- [Explaining Question Answering Models through Text Generation](https://mlsanity.com/paper/2004.05569/) (2020)
- [TensorLog: A Differentiable Deductive Database](https://mlsanity.com/paper/1605.06523/) (2016)
- [Variational Knowledge Graph Reasoning](https://mlsanity.com/paper/1803.06581/) (2018)
- [NELLIE: A Neuro-Symbolic Inference Engine for Grounded, Compositional, and Explainable Reasoning](https://mlsanity.com/paper/2209.07662/) (2022)
- [Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering](https://mlsanity.com/paper/2210.16495/) (2022)
- [Neural Unification for Logic Reasoning over Natural Language](https://mlsanity.com/paper/2109.08460/) (2021)

## Built on by (most cited first, 12 of 12)

- [Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generation](https://mlsanity.com/paper/2405.15307/) (2024)
- [Semantic-Aware Logical Reasoning via a Semiotic Framework](https://mlsanity.com/paper/2509.24765/) (2025)
- [PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving](https://mlsanity.com/paper/2502.16111/) (2025)
- [Asking Before Acting: Gather Information in Embodied Decision Making with Language Models](https://mlsanity.com/paper/2305.15695/) (2023)
- [LASP: Surveying the State-of-the-Art in Large Language Model-Assisted AI Planning](https://mlsanity.com/paper/2409.01806/) (2024)
- [Certified Deductive Reasoning with Language Models](https://mlsanity.com/paper/2306.04031/) (2023)
- [When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives](https://mlsanity.com/paper/2406.12084/) (2024)
- [From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models](https://mlsanity.com/paper/2311.06754/) (2023)
- [Deep Natural Language Feature Learning for Interpretable Prediction](https://mlsanity.com/paper/2311.05754/) (2023)
- [Semiotic logical hexagon theory for LLM logical reasoning](https://mlsanity.com/paper/2607.21933/) (2026)
- [Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation](https://mlsanity.com/paper/2502.19907/) (2025)
- [Scaling Laws for State Dynamics in Large Language Models](https://mlsanity.com/paper/2505.14892/) (2025)

## Abstract (arXiv)

Language models have been shown to perform remarkably well on a wide range of natural language processing tasks. In this paper, we propose LEAP, a novel system that uses language models to perform multi-step logical reasoning and incorporates explicit planning into the inference procedure. Explicit planning enables the system to make more informed reasoning decisions at each step by looking ahead into their future effects. Moreover, we propose a training strategy that safeguards the planning process from being led astray by spurious features. Our full system significantly outperforms other competing methods on multiple standard datasets. When using small T5 models as its core selection and deduction components, our system performs competitively compared to GPT-3 despite having only about 1B parameters (i.e., 175 times smaller than GPT-3). When using GPT-3.5, it significantly outperforms chain-of-thought prompting on the challenging PrOntoQA dataset. We have conducted extensive empirical studies to demonstrate that explicit planning plays a crucial role in the system's performance.