mlsanityData to 14 Oct 2026

Non-myopic Generation of Language Models for Reasoning and Planning

arXiv 2410.17195 ↗First posted 22 Oct 2024cs.AI cs.CLcited by 26

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large Language Models have demonstrated remarkable abilities in reasoning and planning by breaking down complex problems into sequential steps. Despite their success in various domains like mathematical problem-solving and coding, LLMs face challenges in ensuring reliable and optimal planning due to their inherent myopic nature of autoregressive decoding. This paper revisits LLM reasoning from an optimal-control perspective, proposing a novel method, Predictive-Decoding, that leverages Model Predictive Control to enhance planning accuracy. By re-weighting LLM distributions based on foresight trajectories, Predictive-Decoding aims to mitigate early errors and promote non-myopic planning. Our experiments show significant improvements in a wide range of tasks for math, coding, and agents. Furthermore, Predictive-Decoding demonstrates computational efficiency, outperforming search baselines with reduced computational resources. This study provides insights into optimizing LLM planning capabilities.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 41 of its 64 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

41 of 64 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models6 Oct
  2. 2023Reasoning with Language Model is Planning with World Model24 May

Other references · 39

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2021Program Synthesis with Large Language Modelscited by 3,637
  10. 2023Mistral 7Bcited by 3,164
  11. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  12. 2023Code Llama: Open Foundation Models for Codecited by 2,139
27 more
  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  3. 2018World Modelscited by 1,513
  4. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  7. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  9. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  10. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  11. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  12. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 290
  13. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  14. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  15. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
  16. 2024Chain-of-Thought Reasoning Without Promptingcited by 210
  17. 2024The pitfalls of next-token predictioncited by 160
  18. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  19. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 105
  20. 2020Residual Energy-Based Models for Text Generationcited by 95
  21. 2024Do language models plan ahead for future tokens?cited by 49
  22. 2024When is Tree Search Useful for LLM Planning? It Depends on the Discriminatorcited by 47
  23. 2024Learning Iterative Reasoning through Energy Diffusioncited by 36
  24. 2024LiteSearch: Efficacious Tree Search for LLMcited by 31
  25. 2024Empowering Large Language Model Agents through Action Learningcited by 22
  26. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  27. 2023Self-Infilling Code Generationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 11.

26 papers.

Sort:

By area:

  1. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  2. 2025Efficient Reasoning Models: A Surveycited by 95
  3. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  4. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
  5. 2025Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoningcited by 24
  6. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficientcited by 24
  7. 2026Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agentscited by 23
  8. 2025MUR: Momentum Uncertainty guided Reasoning for Large Language Modelscited by 17
  9. 2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitationcited by 13
  10. 2026RelayLLM: Efficient Reasoning via Collaborative Decodingcited by 8
  11. 2025CoFFT: Chain of Foresight-Focus Thought for Visual Language Modelscited by 6
  12. 2025Semantic Probabilistic Control of Language Modelscited by 4
14 more
  1. 2025POSS: Position Specialist Generates Better Draft for Speculative Decodingcited by 2
  2. 2026Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decodingcited by 2
  3. 2026Finding the Cracks: Improving LLMs Reasoning with Paraphrastic Probing and Consistency Verificationcited by 2
  4. 2025Efficient Thought Space Exploration Through Strategic Interventioncited by 1
  5. 2025Mini Amusement Parks (MAPs): A Testbed for Modelling Business Decisionscited by 1
  6. 2026MAXS: Meta-Adaptive Exploration with LLM Agentscited by 1
  7. 2026Martingale Foresight Sampling: A Principled Approach to Inference-Time LLM Decodingcited by 1
  8. 2026Decoding in Geometry: Alleviating Embedding-Space Crowding for Complex Reasoningcited by 1
  9. 2026Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scalingcited by 1
  10. 2025Planning with Diffusion Models for Target-Oriented Dialogue Systemscited by 0
  11. 2025Keyword-Centric Prompting for One-Shot Event Detection with Self-Generated Rationale Enhancementscited by 0
  12. 2025ToMPO: Training LLM Strategic Decision Making from a Multi-Agent Perspectivecited by 0
  13. 2026DeepLook: Deeper Thinking with Lookaheadcited by 0
  14. 2026From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)
  2. 2025Large Language Models for Planning: A Comprehensive and Systematic Survey
  3. 2025LLMs Can Plan Only If We Tell Them
  4. 2024Thinking Forward and Backward: Effective Backward Planning with Large Language Models
  5. 2023On the Planning Abilities of Large Language Models : A Critical Investigation
  6. 2026Towards Reliable and Robust LLM Planning: Symbolic Feedback-Driven Iterative Self-Refinement Framework
  7. 2026Limited Reasoning Space: The cage of long-horizon reasoning in LLMs
  8. 2026Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning