mlsanityData to 7 Oct 2026

Guiding Language Model Reasoning with Planning Tokens

arXiv 2310.05707 ↗First posted 9 Oct 2023cs.CL cs.AI cs.LGcited by 58

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) have recently attracted considerable interest for their ability to perform complex reasoning tasks, such as chain-of-thought (CoT) reasoning. However, most of the existing approaches to enhance this ability rely heavily on data-driven methods, while neglecting the structural aspects of the model's reasoning capacity. To encourage a more structural generation of CoT steps, we propose a hierarchical generation scheme: we let the LM generate a planning token at the start of each reasoning step, intuitively serving as a high-level plan of the current step, and add their embeddings to the model parameters. Our approach requires a negligible increase in trainable parameters (0.001%) and can be applied through either full fine-tuning or a more parameter-efficient scheme. We demonstrate our method's effectiveness by applying it to three different LLMs, showing notable accuracy improvements across three math word problem datasets and one multihop QA dataset with respect to standard fine-tuning baselines.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 58 of its 78 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

58 of 78 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective24 May
  2. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning23 May
  3. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 52

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2013Auto-Encoding Variational Bayescited by 3,980
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
38 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  2. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  4. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  5. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  6. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  7. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  8. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  9. 2023Vision Transformers Need Registerscited by 844
  10. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  11. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  12. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  13. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  14. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  15. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  16. 2023Textbooks Are All You Needcited by 561
  17. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  18. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  19. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  20. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  21. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  22. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  23. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  24. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  25. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  26. 2022Recurrent Memory Transformercited by 197
  27. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  28. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  29. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  30. 2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
  31. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  32. 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
  33. 2017Hard Mixtures of Experts for Large Scale Weakly Supervised Visioncited by 67
  34. 2021Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Promptscited by 64
  35. 2023Scaling Expert Language Models with Unsupervised Domain Discoverycited by 59
  36. 2017Z-Forcing: Training Stochastic Recurrent Networkscited by 51
  37. 2023Adapting Large Language Models to Domains via Reading Comprehensioncited by 48
  38. 2021Efficient Hierarchical Domain Adaptation for Pretrained Language Modelscited by 32

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 34; 2026: 17.

58 papers.

Sort:

By area:

  1. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  4. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  5. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 120
  6. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  7. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  8. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  9. 2025Efficient Reasoning Models: A Surveycited by 93
  10. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 87
  11. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 73
  12. 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifierscited by 58
38 more
  1. 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
  2. 2024Disentangling Memory and Reasoning Ability in Large Language Modelscited by 43
  3. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  4. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  5. 2025Introducing Visual Perception Token into Multimodal Large Language Modelcited by 26
  6. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  7. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  8. 2025Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Thinkcited by 16
  9. 2025Multi-Token Prediction Needs Registerscited by 16
  10. 2025Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thoughtcited by 16
  11. 2025PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Spacecited by 14
  12. 2025GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projectioncited by 13
  13. 2025Enhancing Latent Computation in Transformers with Latent Tokenscited by 13
  14. 2024Learning to Plan for Language Modeling from Unlabeled Datacited by 12
  15. 2025Controlling Thinking Speed in Reasoning Modelscited by 10
  16. 2025Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief Statescited by 9
  17. 2025Efficient LLM Collaboration via Planningcited by 7
  18. 2026Show, Don't Tell: Morphing Latent Reasoning into Image Generationcited by 7
  19. 2024Large Language Models are Limited in Out-of-Context Knowledge Reasoningcited by 6
  20. 2026SeLaR: Selective Latent Reasoning in Large Language Modelscited by 6
  21. 2025Learning to Ponder: Adaptive Reasoning in Latent Spacecited by 4
  22. 2026Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalizationcited by 4
  23. 2026OneLatent: Single-Token Compression for Visual Latent Reasoningcited by 4
  24. 2026DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoningcited by 4
  25. 2025Learning a Continue-Thinking Token for Enhanced Test-Time Scalingcited by 3
  26. 2026Pretraining with Token-Level Adaptive Latent Chain-of-Thoughtcited by 3
  27. 2026Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoningcited by 3
  28. 2026PonderLM-3: Adaptive Token-Wise Pondering with Differentiable Maskingcited by 3
  29. 2026PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Modelscited by 3
  30. 2026Continuous Latent Contexts Enable Efficient Online Learning in Transformerscited by 3
  31. 2025Think Socially via Cognitive Reasoningcited by 2
  32. 2026Plan, Verify and Fill: A Structured Parallel Decoding Approach for Diffusion Language Modelscited by 2
  33. 2026Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoningcited by 2
  34. 2025PD3F: A Pluggable and Dynamic DoS-Defense Framework Against Resource Consumption Attacks Targeting Large Language Modelscited by 1
  35. 2025Token-Controlled Re-ranking for Sequential Recommendation via LLMscited by 1
  36. 2026Thinking into the Future: Latent Lookahead Training for Transformerscited by 1
  37. 2026Transformers Provably Learn to Internalize Chain-of-Thoughtcited by 1
  38. 2024End-to-end Planner Training for Language Modelingcited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency
  2. 2025Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates
  3. 2025CRISP: Complex Reasoning with Interpretable Step-based Plans
  4. 2023Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning
  5. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  6. 2025Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding
  7. 2024Multi-Step Reasoning with Large Language Models, a Survey
  8. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance