mlsanityData to 7 Oct 2026

Dynamic Parallel Tree Search for Efficient LLM Reasoning

arXiv 2502.16235 ↗First posted 22 Feb 2025cs.AIcited by 42

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Tree of Thoughts (ToT) enhances Large Language Model (LLM) reasoning by structuring problem-solving as a spanning tree. However, recent methods focus on search accuracy while overlooking computational efficiency. The challenges of accelerating the ToT lie in the frequent switching of reasoning focus, and the redundant exploration of suboptimal solutions. To alleviate this dilemma, we propose Dynamic Parallel Tree Search (DPTS), a novel parallelism framework that aims to dynamically optimize the reasoning path in inference. It includes the Parallelism Streamline in the generation phase to build up a flexible and adaptive parallelism with arbitrary paths by fine-grained cache management and alignment. Meanwhile, the Search and Transition Mechanism filters potential candidates to dynamically maintain the reasoning focus on more possible solutions and have less redundancy. Experiments on Qwen-2.5 and Llama-3 with Math500 and GSM8K datasets show that DPTS significantly improves efficiency by 2-4x on average while maintaining or even surpassing existing reasoning algorithms in accuracy, making ToT-based reasoning more scalable and computationally efficient.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 35 of its 47 references that are in the map. First posted on arXiv 22 Feb 2025.

Builds on

35 of 47 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
  2. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning12 Dec
  3. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions21 Nov
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  5. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug
  6. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost29 Jul
  7. 2024V-STaR: Training Verifiers for Self-Taught Reasoners9 Feb
  8. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  9. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep
  10. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation28 Jul

Other references · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  12. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
13 more
  1. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  2. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  3. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  4. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 295
  5. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 238
  6. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 156
  7. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  8. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95
  9. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 80
  10. 2024Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizingcited by 49
  11. 2024Large Language Model Inference Acceleration: A Comprehensive Hardware Perspectivecited by 36
  12. 2024Theorem-Validated Reverse Chain-of-Thought Problem Generation for Geometric Reasoningcited by 32
  13. 2023Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Designcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 12.

42 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  4. 2025Efficient Reasoning Models: A Surveycited by 93
  5. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attentioncited by 32
  6. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  7. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  8. 2025Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Trainingcited by 16
  9. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  10. 2025Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Modelscited by 13
  11. 2025AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Controlcited by 11
  12. 2025Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoningcited by 11
30 more
  1. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Modelscited by 11
  2. 2025Exploring the System 1 Thinking Capability of Large Reasoning Modelscited by 10
  3. 2025Optimizing Diversity and Quality through Base-Aligned Model Collaborationcited by 8
  4. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 7
  5. 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoringcited by 6
  6. 2025Supervised Optimism Correction: Be Confident When LLMs Are Surecited by 5
  7. 2025PREMISE: Scalable and Strategic Prompt Optimization for Efficient Mathematical Reasoning in Large Modelscited by 5
  8. 2025CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMscited by 5
  9. 2026InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seekingcited by 5
  10. 2025LogicOCR: Do Your Large Multimodal Models Excel at Logical Reasoning on Text-Rich Images?cited by 4
  11. 2025SSR: Speculative Parallel Scaling Reasoning in Test-timecited by 4
  12. 2025The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Mergingcited by 4
  13. 2025Efficient Tree-Structured Deep Research with Adaptive Resource Allocationcited by 4
  14. 2025Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scalingcited by 4
  15. 2025Can Past Experience Accelerate LLM Reasoning?cited by 3
  16. 2025OptScale: Probabilistic Optimality for Inference-time Scalingcited by 3
  17. 2026ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoningcited by 3
  18. 2025Mitigating Overthinking through Reasoning Shapingcited by 2
  19. 2025Chopping Trees: Semantic Similarity Based Dynamic Pruning for Tree-of-Thought Reasoningcited by 1
  20. 2026On the Overscaling Curse of Parallel Thinking: System Efficacy Contradicts Sample Efficiencycited by 1
  21. 2026FluxLite: Inference-Time Proposal Control for Discrete Diffusion Modelscited by 1
  22. 2025MARCO: Meta-Reflection with Cross-Referencing for Code Reasoningcited by 0
  23. 2025Lateral Tree-of-Thoughts Surpasses ToT by Incorporating Logically-Consistent, Low-Utility Candidatescited by 0
  24. 2025Active Confusion Expression in Large Language Models: Leveraging World Models toward Better Social Reasoningcited by 0
  25. 2026Scaling-Aware Adapter for Structure-Grounded LLM Reasoningcited by 0
  26. 2026HyPER: Bridging Exploration and Exploitation for Scalable LLM Reasoning with Hypothesis Path Expansion and Reductioncited by 0
  27. 2026MedVerse: Efficient and Reliable Medical Reasoning via DAG-Structured Parallel Executioncited by 0
  28. 2026Domain-Specialized Tree of Thought through Plug-and-Play Predictorscited by 0
  29. 2026Hyper Algorithm Design Agent: Evolving Learnable Optimizer from Zerocited by 0
  30. 2026SemOPT: Fixing Semantic Errors in LLM-based Optimization Modeling via Reward-Guided Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual Information
  2. 2025Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search
  3. 2026Beyond Fixed Budgets: Characterizing the Inelasticity and Limitations of Tree-of-Thought Reasoning Strategies
  4. 2026When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models
  5. 2026Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Exploration
  6. 2026Tree of Thoughts as a Classical Heuristic Search Problem: Formal Foundations and Design Patterns
  7. 2025Accelerating Large Language Model Reasoning via Speculative Search
  8. 2025A Survey on Parallel Reasoning