mlsanityData to 7 Oct 2026

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning

arXiv 2412.09078 ↗First posted 12 Dec 2024cs.CL cs.AIcited by 110

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable abilities across various language tasks, but solving complex reasoning problems remains a significant challenge. While existing methods, such as Chain-of-Thought (CoT) and Tree-of-Thought (ToT), enhance reasoning by decomposing problems or structuring prompts, they typically perform a single pass of reasoning and may fail to revisit flawed paths, compromising accuracy. To address this limitation, we propose a novel reasoning framework called Forest-of-Thought (FoT), which integrates multiple reasoning trees to leverage collective decision-making for solving complex logical problems. FoT employs sparse activation strategies to select the most relevant reasoning paths, improving both efficiency and accuracy. Additionally, we introduce a dynamic self-correction strategy that enables real-time error correction, along with consensus-guided decision-making strategies to optimize both correctness and computational resources. Experimental results demonstrate that the FoT framework, combined with these strategies, significantly enhances the reasoning capabilities of LLMs, enabling them to solve complex tasks with greater precision and efficiency. Code will be available at https://github.com/iamhankai/Forest-of-Thought.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 19 of its 29 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

19 of 29 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models20 Aug
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 16

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  7. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  8. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  9. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  10. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  11. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 156
  12. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
4 more
  1. 2023Chain of Code: Reasoning with a Language Model-Augmented Code Emulatorcited by 120
  2. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  3. 2023Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generationcited by 73
  4. 2023CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Taskscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 76; 2026: 34.

110 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  4. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 158
  5. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  6. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  7. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  8. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 74
  9. 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoningcited by 69
  10. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 61
  11. 2025Atom of Thoughts for Markov LLM Test-Time Scalingcited by 54
  12. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
38 more
  1. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Modelscited by 51
  2. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoningcited by 42
  3. 2025Reasoning Language Models: A Blueprintcited by 41
  4. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  5. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaoscited by 39
  6. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 32
  7. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  8. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoTcited by 24
  9. 2025Entropy-Gated Branching for Efficient Test-Time Reasoningcited by 21
  10. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  11. 2025ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancycited by 17
  12. 2025DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real Worldcited by 16
  13. 2025Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scalingcited by 15
  14. 2026Learning Global Hypothesis Space for Enhancing Synergistic Reasoning Chaincited by 15
  15. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  16. 2025HyperTree Planning: Enhancing LLM Reasoning via Hierarchical Thinkingcited by 14
  17. 2026Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probingcited by 13
  18. 2025Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Needcited by 12
  19. 2025Learning from Diverse Reasoning Paths with Routing and Collaborationcited by 12
  20. 2025DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Searchcited by 11
  21. 2025Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theorycited by 10
  22. 2025Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelismcited by 10
  23. 2025SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Modelscited by 10
  24. 2025Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Timecited by 10
  25. 2025Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMscited by 9
  26. 2025OWL: Geometry-Aware Spatial Reasoning for Audio Large Language Modelscited by 9
  27. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 8
  28. 2025CyberV: Cybernetics for Test-time Scaling in Video Understandingcited by 8
  29. 2025Large Language Model-Enhanced Multi-Armed Banditscited by 7
  30. 2025AutoChemSchematic AI: Agentic Physics-Aware Automation for Chemical Manufacturing Scale-Upcited by 6
  31. 2025Meta-R1: Empowering Large Reasoning Models with Metacognitioncited by 6
  32. 2025SSR: Socratic Self-Refine for Large Language Model Reasoningcited by 6
  33. 2025Decoding Large Language Diffusion Models with Foreseeing Movementcited by 6
  34. 2026Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clusteringcited by 6
  35. 2025LLM-First Search: Self-Guided Exploration of the Solution Spacecited by 5
  36. 2025Fixing the Broken Compass: Diagnosing and Improving Inference-Time Reward Modelingcited by 4
  37. 2025A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directionscited by 4
  38. 2025Structured Pruning for Diverse Best-of-N Reasoning Optimizationcited by 4

50 most cited of 110 All 110 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Empirical Complexity of Reasoning and Planning in LLMs
  2. 2025MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual Information
  3. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning
  4. 2024Enhancing LLM Reasoning with Reward-guided Tree Search
  5. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  6. 2025Adaptive Graph of Thoughts: Test-Time Adaptive Reasoning Unifying Chain, Tree, and Graph Structures
  7. 2025Chain or tree? Re-evaluating complex reasoning from the perspective of a matrix of thought
  8. 2025Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately