mlsanityData to 14 Oct 2026

Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoning

arXiv 2310.03249 ↗First posted 5 Oct 2023cs.CLcited by 49

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Large language models (LLMs) have achieved remarkable success across a wide spectrum of tasks; however, they still face limitations in scenarios that demand long-term planning and spatial reasoning. To facilitate this line of research, in this work, we propose a new benchmark, termed Path Planning from Natural Language (PPNL). Our benchmark evaluates LLMs' spatial-temporal reasoning by formulating ''path planning'' tasks that require an LLM to navigate to target locations while avoiding obstacles and adhering to constraints. Leveraging this benchmark, we systematically investigate LLMs including GPT-4 via different few-shot prompting methodologies as well as BART and T5 of various sizes via fine-tuning. Our experimental results show the promise of few-shot GPT-4 in spatial reasoning, when it is prompted to reason and act interleavedly, although it still fails to perform long-term temporal reasoning. In contrast, while fine-tuned LLMs achieved impressive results on in-distribution reasoning tasks, they struggled to generalize to larger environments or environments with more obstacles.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 53 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

31 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Task and Motion Planning with Large Language Models for Object Rearrangement10 Mar
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Models12 Jul

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  10. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  12. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
17 more
  1. 2023Large Language Models as Optimizerscited by 864
  2. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  3. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  4. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  5. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  6. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
  7. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
  8. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 133
  9. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 119
  10. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 113
  11. 2020A Benchmark for Systematic Generalization in Grounded Language Understandingcited by 108
  12. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 104
  13. 2019A Theoretical Analysis of the Number of Shots in Few-Shot Learningcited by 39
  14. 2023Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Followingcited by 33
  15. 2022Neuro-Symbolic Procedural Planning with Commonsense Promptingcited by 28
  16. 2023Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniquescited by 24
  17. 2021ReaSCAN: Compositional Reasoning in Language Groundingcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 12; 2025: 25; 2026: 10.

49 papers.

Sort:

By area:

  1. 2024AI Agents That Mattercited by 219
  2. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 83
  3. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 83
  4. 2024VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMscited by 47
  5. 2024LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planningcited by 45
  6. 2024System-1.x: Learning to Balance Fast and Slow Planning with Language Modelscited by 35
  7. 2023LLM A*: Human in the Loop Large Language Models Enabled A* Search for Roboticscited by 26
  8. 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMscited by 25
  9. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  10. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  11. 2025Agents of Change: Self-Evolving LLM Agents for Strategic Planningcited by 21
  12. 2025WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planningcited by 20
37 more
  1. 2024Chain-of-History Reasoning for Temporal Knowledge Graph Forecastingcited by 17
  2. 2025A Survey on Large Language Models for Automated Planningcited by 17
  3. 2025I2-NeRF: Learning Neural Radiance Fields Under Physically-Grounded Media Interactionscited by 17
  4. 2024Evaluating Large Language Models on Spatial Tasks: A Multi-Task Benchmarking Studycited by 12
  5. 2024Can LLM be a Good Path Planner based on Prompt Engineering? Mitigating the Hallucination for Path Planningcited by 11
  6. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 11
  7. 2025From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Taskcited by 10
  8. 2025MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Modelscited by 8
  9. 2024GRASP: A Grid-Based Benchmark for Evaluating Commonsense Spatial Reasoningcited by 7
  10. 2025PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilitiescited by 7
  11. 2024Evaluating Vision-Language Models as Evaluators in Path Planningcited by 6
  12. 2025Large Language Models in Operations Research: Methods, Applications, and Challengescited by 6
  13. 2025SPaRC: A Spatial Pathfinding Reasoning Challengecited by 5
  14. 2025LLMAP: LLM-Assisted Multi-Objective Route Planning with User Preferencescited by 5
  15. 2024Program Synthesis Benchmark for Visual Programming in XLogoOnline Environmentcited by 4
  16. 2025GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environmentscited by 4
  17. 2025Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimationcited by 4
  18. 2025Gricean Norms as a Basis for Effective Collaborationcited by 3
  19. 2025OBSR: Open Benchmark for Spatial Representationscited by 3
  20. 2025Constraint-Aware Route Recommendation from Natural Language via Hierarchical LLM Agentscited by 3
  21. 2023Is Feedback All You Need? Leveraging Natural Language Feedback in Goal-Conditioned Reinforcement Learningcited by 2
  22. 2025Navigating Motion Agents in Dynamic and Cluttered Environments through LLM Reasoningcited by 2
  23. 2025LLM-BabyBench: Can Language Models Plan in Worlds They Can Simulate?cited by 2
  24. 2026The Scenic Route to Deception: Dark Patterns and Explainability Pitfalls in Conversational Navigationcited by 1
  25. 2026Mind the Gap Between Spatial Reasoning and Acting! Step-by-Step Evaluation of Agents With Spatial-Gymcited by 1
  26. 2026STAR: Failure-Aware Markovian Routing for Multi-Agent Spatiotemporal Reasoningcited by 1
  27. 2025Evaluating Spatiotemporal Consistency in Automatically Generated Sewing Instructionscited by 0
  28. 2025CompassLLM: A Multi-Agent Approach toward Geo-Spatial Reasoning for Popular Path Querycited by 0
  29. 2025Iti-Validator: A Guardrail Framework for Validating and Correcting LLM-Generated Itinerariescited by 0
  30. 2025Don't Just Search, Understand: Semantic Path Planning Agent for Spherical Tensegrity Robots in Unknown Environmentscited by 0
  31. 2026MMP-A*: Multimodal Perception Enhanced Incremental Heuristic Search on Path Planningcited by 0
  32. 2026Grid Spatial Understanding: A Dataset for Textual Spatial Reasoning over Grids, Embodied Settings, and Coordinate Structurescited by 0
  33. 2026ItinBench: Benchmarking Planning Across Multiple Cognitive Dimensions with Large Language Modelscited by 0
  34. 2026Planning with Transformers: Chain of Computation and Structured Context Windowscited by 0
  35. 2026OmniRouting: A Semantic-Coupled Multimodal Benchmark for Constraint-Aware Spatial Reasoning in PCB Routingcited by 0
  36. 2026GeoBenchLLM: A Comprehensive Benchmark for Evaluating LLMs on Geo-Related Taskscited by 0
  37. 2026Is Visual Prompting All You Need? Studying VLM Spatial Reasoning under Progressive Visual Scaffoldscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks
  2. 2024Can Large Language Models Create New Knowledge for Spatial Reasoning Tasks?
  3. 2025Stuck in the Matrix: Probing Spatial Reasoning in Large Language Models
  4. 2026SokoBench: Evaluating Long-Horizon Planning and Reasoning in Large Language Models
  5. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
  6. 2024Exploring and Benchmarking the Planning Capabilities of Large Language Models
  7. 2024Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark
  8. 2026Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models