mlsanityData to 14 Oct 2026

When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models

arXiv 2608.30395 ↗First posted 31 Aug 2026cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

As pretraining scaling laws approach saturation, Test-Time Scaling (TTS) has emerged as an important direction for improving reasoning by allocating inference-time compute to a fixed model prior. Viewed at a high level, TTS reframes inference as search over a space of partial reasoning states. While Chain-of-Thought (CoT) exposes intermediate steps, common instantiations rely on single-trajectory decoding, limiting recovery from early errors and exploration. This survey systematizes recent progress in tree-search-based reasoning, viewing inference as instance-specific optimization rather than decoding. We trace the evolution from uninformed search to Monte Carlo Tree Search (MCTS), highlighting how sampling-based control supports principled exploration-exploitation trade-offs. To unify a fragmented literature, we introduce a Unified Design Space spanning search topology, evaluation signals, and control dynamics, and advocate a standardized compute-reporting abstraction to make compute-accuracy trade-offs explicit and comparable.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 170 of its 241 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

170 of 241 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning5 May
  2. 2025Unifying Tree Search Algorithm and Reward Design for LLM Reasoning: A Survey11 Oct
  3. 2025Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTS8 Jul
  4. 2025SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation6 Jun
  5. 2025LLM-First Search: Self-Guided Exploration of the Solution Space5 Jun
  6. 2025Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning6 Apr
  7. 2025CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning4 Feb
  8. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design15 Jan
  9. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  10. 2024Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling19 Dec
  11. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective18 Dec
  12. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions21 Nov
6 more
  1. 2024O1 Replication Journey: A Strategic Progress Report -- Part 18 Oct
  2. 2024Interpretable Contrastive Monte Carlo Tree Search Reasoning2 Oct
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  4. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing18 Apr
  5. 2023Autonomous Tree-search Ability of Large Language Models14 Oct
  6. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep

Other references · 152

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  7. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  8. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  9. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  10. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  11. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  12. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
38 more
  1. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  2. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  3. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  4. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  5. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  6. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
  7. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 208
  8. 2023Planning with Large Language Models for Code Generationcited by 182
  9. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
  10. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 158
  11. 2024Tree Search for Language Model Agentscited by 151
  12. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 145
  13. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 138
  14. 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinementcited by 132
  15. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  16. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 109
  17. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 105
  18. 2024ARGS: Alignment as Reward-Guided Searchcited by 103
  19. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 102
  20. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 97
  21. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 91
  22. 2023OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoningcited by 85
  23. 2025BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Provingcited by 84
  24. 2024Planning In Natural Language Improves LLM Search For Code Generationcited by 83
  25. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 82
  26. 2024o1-Coder: an o1 Replication for Codingcited by 78
  27. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 77
  28. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  29. 2023Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generationcited by 74
  30. 2024Step-level Value Preference Optimization for Mathematical Reasoningcited by 72
  31. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 69
  32. 2023Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planningcited by 61
  33. 2023Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decodingcited by 60
  34. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTScited by 54
  35. 2024Efficient Causal Graph Discovery Using Large Language Modelscited by 53
  36. 2025SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolutioncited by 53
  37. 2024Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervisioncited by 52
  38. 2024When is Tree Search Useful for LLM Planning? It Depends on the Discriminatorcited by 47

50 most cited of 152 All 152 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search
  2. 2026Beyond Fixed Budgets: Characterizing the Inelasticity and Limitations of Tree-of-Thought Reasoning Strategies
  3. 2026Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
  4. 2025The Art of Scaling Test-Time Compute for Large Language Models
  5. 2025Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search
  6. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification
  7. 2025Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search
  8. 2025A*-Decoding: Token-Efficient Inference Scaling