mlsanityData to 7 Oct 2026

Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B

arXiv 2406.07394 ↗First posted 11 Jun 2024cs.AIcited by 156

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

This paper introduces the MCT Self-Refine (MCTSr) algorithm, an innovative integration of Large Language Models (LLMs) with Monte Carlo Tree Search (MCTS), designed to enhance performance in complex mathematical reasoning tasks. Addressing the challenges of accuracy and reliability in LLMs, particularly in strategic and mathematical reasoning, MCTSr leverages systematic exploration and heuristic self-refine mechanisms to improve decision-making frameworks within LLMs. The algorithm constructs a Monte Carlo search tree through iterative processes of Selection, self-refine, self-evaluation, and Backpropagation, utilizing an improved Upper Confidence Bound (UCB) formula to optimize the exploration-exploitation balance. Extensive experiments demonstrate MCTSr's efficacy in solving Olympiad-level mathematical problems, significantly improving success rates across multiple datasets, including GSM8K, GSM Hard, MATH, and Olympiad-level benchmarks, including Math Odyssey, AIME, and OlympiadBench. The study advances the application of LLMs in complex reasoning tasks and sets a foundation for future AI integration, enhancing decision-making accuracy and reliability in LLM-driven applications.

Key facts. Cited by 156 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 28 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

22 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023No Train Still Gain. Unleash Mathematical Reasoning of Large Language Models with Monte Carlo Tree Search Guided by Energy Function1 Sep
  2. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct18 Aug
  3. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Models3 Aug

Other references · 19

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  10. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  11. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  12. 2022PAL: Program-aided Language Modelscited by 635
7 more
  1. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  2. 2023LLM-Rec: Personalized Recommendation via Prompting Large Language Modelscited by 77
  3. 2023DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphscited by 40
  4. 2023Monte-Carlo Tree Search for Multi-Agent Pathfinding: Preliminary Resultscited by 9
  5. 2024PhyPlan: Compositional and Adaptive Physical Task Reasoning with Physics-Informed Skill Networks for Robot Manipulatorscited by 6
  6. 2023An Integrated Framework Integrating Monte Carlo Tree Search and Supervised Learning for Train Timetabling Problemcited by 5
  7. 2023General Method for Solving Four Types of SAT Problemscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 26; 2025: 100; 2026: 30.

156 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  4. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 167
  5. 2024Tree Search for Language Model Agentscited by 148
  6. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 137
  7. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 118
  8. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 113
  9. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  10. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoningcited by 110
  11. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Explorationcited by 100
  12. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 100
38 more
  1. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  2. 2024Interpretable Contrastive Monte Carlo Tree Search Reasoningcited by 62
  3. 2025Parallel-R1: Towards Parallel Thinking via Reinforcement Learningcited by 59
  4. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTScited by 52
  5. 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoningcited by 50
  6. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  7. 2024Autoformulation of Mathematical Optimization Models Using LLMscited by 48
  8. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoningcited by 42
  9. 2025Reasoning Language Models: A Blueprintcited by 41
  10. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 40
  11. 2024Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verificationcited by 40
  12. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  13. 2025MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verificationcited by 39
  14. 2024Entropy-Regularized Process Reward Modelcited by 38
  15. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  16. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 36
  17. 2025ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verificationcited by 35
  18. 2025Monte Carlo Tree Diffusion for System 2 Planningcited by 33
  19. 2025MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluationcited by 32
  20. 2025Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generationcited by 30
  21. 2026Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpeningcited by 27
  22. 2025On the Emergence of Thinking in LLMs I: Searching for the Right Intuitioncited by 22
  23. 2024An LLM Agent for Automatic Geospatial Data Analysiscited by 21
  24. 2025DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversalcited by 20
  25. 2025Thought calibration: Efficient and confident test-time scalingcited by 20
  26. 2025BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoningcited by 19
  27. 2025Scaling Autonomous Agents via Automatic Reward Modeling And Planningcited by 19
  28. 2025Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agentscited by 19
  29. 2024Towards a Unified View of Preference Learning for Large Language Models: A Surveycited by 17
  30. 2024MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translationcited by 17
  31. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Modelscited by 16
  32. 2025FastMCTS: A Simple Sampling Strategy for Data Synthesiscited by 16
  33. 2025PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solvingcited by 15
  34. 2024VerifierQ: Enhancing LLM Test Time Compute with Q-Learning-based Verifierscited by 14
  35. 2024What Are Step-Level Reward Models Rewarding? Counterintuitive Findings from MCTS-Boosted Mathematical Reasoningcited by 14
  36. 2025ProgCo: Program Helps Self-Correction of Large Language Modelscited by 14
  37. 2025Enhancing Reasoning through Process Supervision with Monte Carlo Tree Searchcited by 14
  38. 2025Reasoning with Reinforced Functional Token Tuningcited by 14

50 most cited of 156 All 156 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning
  2. 2025CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Model
  3. 2025Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
  4. 2025SMRC: Aligning Large Language Models with Student Reasoning for Mathematical Error Correction
  5. 2025Evaluating Frontier LLMs on PhD-Level Mathematical Reasoning: A Benchmark on a Textbook in Theoretical Computer Science about Randomized Algorithms
  6. 2025MASTER: A Multi-Agent System with LLM Specialized MCTS
  7. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data
  8. 2026Beyond Accuracy: Evaluating Strategy Diversity in LLM Mathematical Reasoning