mlsanityData to 7 Oct 2026

A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworks

arXiv 2501.10069 ↗First posted 17 Jan 2025cs.AIcited by 21

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

LLM test-time compute (or LLM inference) via search has emerged as a promising research area with rapid developments. However, current frameworks often adopt distinct perspectives on three key aspects: task definition, LLM profiling, and search procedures, making direct comparisons challenging. Moreover, the search algorithms employed often diverge from standard implementations, and their specific characteristics are not thoroughly specified. This survey aims to provide a comprehensive but integrated technical review on existing LIS frameworks. Specifically, we unify task definitions under Markov Decision Process (MDP) and provides modular definitions of LLM profiling and search procedures. The definitions enable precise comparisons of various LLM inference frameworks while highlighting their departures from conventional search algorithms. We also discuss the applicability, performance, and efficiency of these methods. For ongoing paper updates, please refer to our GitHub repository: https://github.com/xinzhel/LLM-Search.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 70 of its 89 references that are in the map. First posted on arXiv 17 Jan 2025.

Builds on

70 of 89 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023PathFinder: Guided Search over Multi-Step Reasoning Paths8 Dec
  2. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  4. 2023Self-Evaluation Guided Beam Search for Reasoning1 May
  5. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  6. 2022Language Models (Mostly) Know What They Know11 Jul

Other references · 64

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  11. 2023Let's Verify Step by Stepcited by 3,598
  12. 2021Program Synthesis with Large Language Modelscited by 3,509
38 more
  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  3. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  4. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  6. 2021Measuring Coding Challenge Competence With APPScited by 879
  7. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  8. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  9. 2023Whose Opinions Do Language Models Reflect?cited by 759
  10. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  11. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  12. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  13. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  14. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  15. 2024Understanding the planning of LLM agents: A surveycited by 414
  16. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  17. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  18. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  19. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  20. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  21. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  22. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  23. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  24. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  25. 2019Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Goodcited by 207
  26. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  27. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  28. 2023Planning with Large Language Models for Code Generationcited by 179
  29. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 171
  30. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  31. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
  32. 2024Tree Search for Language Model Agentscited by 148
  33. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  34. 2023On the Tool Manipulation Capability of Open-source Large Language Modelscited by 122
  35. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  36. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95
  37. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 94
  38. 2024Planning In Natural Language Improves LLM Search For Code Generationcited by 79

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 6.

21 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  3. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
  4. 2025Entropy-Gated Branching for Efficient Test-Time Reasoningcited by 21
  5. 2025M1: Towards Scalable Test-Time Compute with Mamba Reasoning Modelscited by 21
  6. 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Modelscited by 19
  7. 2025SLOT: Sample-specific Language Model Optimization at Test-timecited by 17
  8. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 13
  9. 2025DeepTrans: Deep Reasoning Translation via Reinforcement Learningcited by 8
  10. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 7
  11. 2025AutoChemSchematic AI: Agentic Physics-Aware Automation for Chemical Manufacturing Scale-Upcited by 6
  12. 2025Do Language Models Follow Occam's Razor? An Evaluation of Parsimony in Inductive and Abductive Reasoningcited by 4
9 more
  1. 2026RF-Agent: Automated Reward Function Design via Language Agent Tree Searchcited by 3
  2. 2025Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspectivecited by 2
  3. 2025KROMA: Ontology Matching with Knowledge Retrieval and Large Language Modelscited by 2
  4. 2025Reinforcement Learning for Long-Horizon Multi-Turn Search Agentscited by 2
  5. 2026Alignment-Weighted DPO: A principled reasoning approach to improve safety alignmentcited by 2
  6. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2
  7. 2025The Virtues of Brevity: Avoid Overthinking in Parallel Test-Time Reasoningcited by 0
  8. 2026Complete Cyclic Subtask Graphs for Tool-Using LLM Agents: Flexibility, Cost, and Bottlenecks in Long-Horizon Workflowscited by 0
  9. 2026CheckRLM: Effective Knowledge-Thought Coherence Checking in Retrieval-Augmented Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
  2. 2026Provable Test-Time Scaling for Beam Search in LLM Reasoning
  3. 2025Unifying Tree Search Algorithm and Reward Design for LLM Reasoning: A Survey
  4. 2025Review of Inference-Time Scaling Strategies: Reasoning, Search and RAG
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
  6. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  7. 2024A Survey on LLM Inference-Time Self-Improvement
  8. 2025Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute