mlsanityData to 7 Oct 2026

Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models

arXiv 2402.11140 ↗First posted 17 Feb 2024cs.CL cs.AI cs.LGcited by 22

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

The reasoning performance of Large Language Models (LLMs) on a wide range of problems critically relies on chain-of-thought prompting, which involves providing a few chain of thought demonstrations as exemplars in prompts. Recent work, e.g., Tree of Thoughts, has pointed out the importance of exploration and self-evaluation in reasoning step selection for complex problem solving. In this paper, we present Boosting of Thoughts (BoT), an automated prompting framework for problem solving with LLMs by iteratively exploring and self-evaluating many trees of thoughts in order to acquire an ensemble of trial-and-error reasoning experiences, which will serve as a new form of prompting to solve the complex problem. Starting from a simple prompt without requiring examples, BoT iteratively explores and evaluates a large collection of reasoning steps, and more importantly, uses error analysis obtained from the LLM on them to explicitly revise prompting, which in turn enhances reasoning step generation, until a final answer is attained. Our experiments with GPT-4 and Llama2 across extensive complex mathematical problems demonstrate that BoT consistently achieves higher or comparable problem-solving rates than other advanced prompting approaches.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 34 references that are in the map. First posted on arXiv 17 Feb 2024.

Builds on

30 of 34 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  2. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Models19 Apr
  3. 2023Boosted Prompt Ensembles for Large Language Models12 Apr
  4. 2023Active Prompting with Chain-of-Thought for Large Language Models23 Feb
  5. 2022Large Language Models are Better Reasoners with Self-Verification19 Dec
  6. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  7. 2022Complexity-Based Prompting for Multi-Step Reasoning3 Oct
  8. 2022Large Language Models are Zero-Shot Reasoners24 May
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  12. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
7 more
  1. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  2. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  3. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  4. 2023Cumulative Reasoning with Large Language Modelscited by 94
  5. 2022PromptBoosting: Black-Box Text Classification with Ten Forward Passescited by 36
  6. 2023Automatic Model Selection with Large Language Models for Reasoningcited by 35
  7. 2023PREFER: Prompt Ensemble Learning via Feedback-Reflect-Refinecited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 11; 2026: 5.

22 papers.

Sort:

By area:

  1. 2024Large Language Models and Games: A Survey and Roadmapcited by 91
  2. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  3. 2024Disentangling Memory and Reasoning Ability in Large Language Modelscited by 43
  4. 2025Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoningcited by 34
  5. 2025Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physicscited by 33
  6. 2024Meta Reasoning for Large Language Modelscited by 29
  7. 2024Instance-adaptive Zero-shot Chain-of-Thought Promptingcited by 25
  8. 2025A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworkscited by 21
  9. 2024Toward Adaptive Reasoning in Large Language Models with Thought Rollbackcited by 11
  10. 2025Fast-Slow-Thinking: Complex Task Solving with Large Language Modelscited by 7
  11. 2025RefCritic: Training Long Chain-of-Thought Critic Models with Refinement Feedbackcited by 6
  12. 2024Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Modelscited by 4
10 more
  1. 2026Finding the Cracks: Improving LLMs Reasoning with Paraphrastic Probing and Consistency Verificationcited by 2
  2. 2025mSCoRe: a Multilingual and Scalable Benchmark for Skill-based Commonsense Reasoningcited by 1
  3. 2025Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compressioncited by 1
  4. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  5. 2025iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planningcited by 1
  6. 2025Lateral Tree-of-Thoughts Surpasses ToT by Incorporating Logically-Consistent, Low-Utility Candidatescited by 0
  7. 2026Computability of Agentic Systemscited by 0
  8. 2026Can MLLMs Read Students' Minds? Unpacking Multimodal Error Analysis in Handwritten Mathcited by 0
  9. 2026LinTree: Improving LLM Reasoning with Explicitly Structured Search Historiescited by 0
  10. 2026Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Abilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought
  2. 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models
  3. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  4. 2023Large Language Models as Analogical Reasoners
  5. 2024Expanding Search Space with Diverse Prompting Agents: An Efficient Sampling Approach for LLM Mathematical Reasoning
  6. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models
  7. 2024Think Beyond Size: Adaptive Prompting for More Effective Reasoning
  8. 2024BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts