mlsanityData to 14 Oct 2026

Tree of Problems: Improving structured problem solving with compositionality

arXiv 2410.06634 ↗First posted 9 Oct 2024cs.CLcited by 3

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable performance across multiple tasks through in-context learning. For complex reasoning tasks that require step-by-step thinking, Chain-of-Thought (CoT) prompting has given impressive results, especially when combined with self-consistency. Nonetheless, some tasks remain particularly difficult for LLMs to solve. Tree of Thoughts (ToT) and Graph of Thoughts (GoT) emerged as alternatives, dividing the complex problem into paths of subproblems. In this paper, we propose Tree of Problems (ToP), a simpler version of ToT, which we hypothesise can work better for complex tasks that can be divided into identical subtasks. Our empirical results show that our approach outperforms ToT and GoT, and in addition performs better than CoT on complex reasoning tasks. All code for this paper is publicly available here: https://github.com/ArmelRandy/tree-of-problems.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 12 of its 18 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

12 of 18 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  2. 2022Complexity-Based Prompting for Multi-Step Reasoning3 Oct
  3. 2022Large Language Models are Zero-Shot Reasoners24 May
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 8

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  6. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  7. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  8. 2022Successive Prompting for Decomposing Complex Questionscited by 94

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translationcited by 14
  2. 2025STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasingscited by 8
  3. 2025PARALLELPROMPT: Extracting Parallelism from Large Language Model Queriescited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  2. 2024On the Empirical Complexity of Reasoning and Planning in LLMs
  3. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
  4. 2023Large Language Model Guided Tree-of-Thought
  5. 2025Structured Reasoning with Tree-of-Thoughts for Bengali Math Word Problems
  6. 2024Demystifying Chains, Trees, and Graphs of Thoughts
  7. 2025Chain or tree? Re-evaluating complex reasoning from the perspective of a matrix of thought
  8. 2026Beyond Fixed Budgets: Characterizing the Inelasticity and Limitations of Tree-of-Thought Reasoning Strategies