mlsanityData to 14 Oct 2026

Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models

arXiv 2305.10276 ↗First posted 17 May 2023cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we take the initiative to investigate the performance of LLMs on complex planning tasks that require LLMs to understand a virtual spatial environment simulated via natural language and act correspondingly in text. We propose a benchmark named Natural Language Planning and Action (Natala) composed of a set of novel tasks: Brick World, NLVR-based Manipulations, and Natural Language Navigation. We found that current popular LLMs such as ChatGPT still lack abilities in complex planning. This arises a question -- do the LLMs have a good understanding of the environments described in natural language, or maybe other alternatives such as symbolic representations are neater and hence better to be understood by LLMs? To this end, we propose a novel method called CoS (Chain-of-Symbol Prompting) that represents the complex environments with condensed symbolic spatial representations during the chained intermediate thinking steps. CoS is easy to use and does not need additional training on LLMs. Extensive experiments indicate that CoS clearly surpasses the performance of the Chain-of-Thought (CoT) Prompting in all three planning tasks with even fewer tokens used in the inputs compared with CoT on ChatGPT and InstructGPT. The performance gain is strong, by up to 60.8% accuracy (from 31.8% to 92.6%) on Brick World for ChatGPT. CoS also reduces the number of tokens in the prompt obviously, by up to 65.8% of the tokens (from 407 to 139) for the intermediate steps from demonstrations on Brick World. Code and data available at: https://github.com/hanxuhu/chain-of-symbol-planning

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 14 of its 23 references that are in the map. First posted on arXiv 17 May 2023.

Builds on

14 of 23 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  5. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  6. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  7. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  8. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  9. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  10. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  11. 2021SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoningcited by 77
  12. 2022Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoningcited by 34
2 more
  1. 2023Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questionscited by 30
  2. 2017Representation Learning for Grounded Spatial Reasoningcited by 23

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 4.

6 papers.

  1. 2024Break the Chain: Large Language Models Can be Shortcut Reasonerscited by 32
  2. 2024Meta Reasoning for Large Language Modelscited by 29
  3. 2025Understanding LLM Scientific Reasoning through Promptings and Model's Explanation on the Answerscited by 10
  4. 2025Enhancing Spatial Reasoning in Vision-Language Models via Chain-of-Thought Prompting and Reinforcement Learningcited by 9
  5. 2025Affordable AI Assistants with Knowledge Graph of Thoughtscited by 6
  6. 2025ThinkPilot: Steering Reasoning Models via Automated Think-prefixes Optimizationcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks
  2. 2025LLMs Can Plan Only If We Tell Them
  3. 2023Dynamic Planning with a LLM
  4. 2023Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoning
  5. 2024Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation
  6. 2023Translating Natural Language to Planning Goals with Large-Language Models
  7. 2026Knowledge Model Prompting Increases LLM Performance on Planning Tasks
  8. 2025MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models