mlsanityData to 7 Oct 2026

Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design

arXiv 2501.08603 ↗First posted 15 Jan 2025cs.AIcited by 100

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Handcrafting heuristics for solving complex optimization tasks (e.g., route planning and task allocation) is a common practice but requires extensive domain knowledge. Recently, Large Language Model (LLM)-based automatic heuristic design (AHD) methods have shown promise in generating high-quality heuristics without manual interventions. Existing LLM-based AHD methods employ a population to maintain a fixed number of top-performing LLM-generated heuristics and introduce evolutionary computation (EC) to iteratively enhance the population. However, these population-based procedures cannot fully develop the potential of each heuristic and are prone to converge into local optima. To more comprehensively explore the space of heuristics, this paper proposes to use Monte Carlo Tree Search (MCTS) for LLM-based heuristic evolution. The proposed MCTS-AHD method organizes all LLM-generated heuristics in a tree structure and can better develop the potential of temporarily underperforming heuristics. In experiments, MCTS-AHD delivers significantly higher-quality heuristics on various complex tasks. Our code is available.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 34 of its 115 references that are in the map. First posted on arXiv 15 Jan 2025.

Builds on

34 of 115 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep

Other references · 33

  1. 2023Large Language Models as Optimizerscited by 830
  2. 2018Attention, Learn to Solve Routing Problems!cited by 646
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 600
  5. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  6. 2024Understanding the planning of LLM agents: A surveycited by 414
  7. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  8. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  9. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 178
  10. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 156
  11. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 137
  12. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 119
21 more
  1. 2023System 2 Attention (is something you might need too)cited by 85
  2. 2021Graph Neural Network Guided Local Search for the Traveling Salesperson Problemcited by 70
  3. 2024HSEvo: Elevating Automatic Heuristic Design with Diversity-Driven Harmony Search and Genetic Algorithm Using LLMscited by 68
  4. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 65
  5. 2024RouteFinder: Towards Foundation Models for Vehicle Routing Problemscited by 54
  6. 2024Ant Colony Sampling with GFlowNets for Combinatorial Optimizationcited by 53
  7. 2023Algorithm Evolution Using Large Language Modelcited by 52
  8. 2024Multi-objective Evolution of Heuristic Using Large Language Modelcited by 52
  9. 2024UDC: A Unified Neural Divide-and-Conquer Framework for Large-Scale Combinatorial Optimization Problemscited by 45
  10. 2024Make Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTScited by 42
  11. 2020Cost-aware Bayesian Optimizationcited by 41
  12. 2024GOAL: A Generalist Combinatorial Optimization Agent Learnercited by 39
  13. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 38
  14. 2023Towards Optimizing with Large Language Modelscited by 34
  15. 2024LLM4AD: A Platform for Algorithm Design with Large Language Modelcited by 33
  16. 2024PyBench: Evaluating LLM Agent on various real-world coding taskscited by 24
  17. 2024DPN: Decoupling Partition and Navigation for Neural Solvers of Min-max Vehicle Routing Problemscited by 18
  18. 2024Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithmscited by 12
  19. 2024LLMs can Schedulecited by 11
  20. 2024Bridging Large Language Models and Optimization: A Unified Framework for Text-attributed Combinatorial Optimizationcited by 8
  21. 2024Controlling the Mutation in Large Language Models for the Efficient Evolution of Algorithmscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 39; 2026: 61.

100 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025EoH-S: Evolution of Heuristic Set using LLMs for Automated Heuristic Designcited by 32
  3. 2025CO-Bench: Benchmarking Language Model Agents in Algorithm Search for Combinatorial Optimizationcited by 28
  4. 2025Large Language Models as End-to-end Combinatorial Optimization Solverscited by 27
  5. 2025StepORLM: A Self-Evolving Framework With Generative Process Supervision For Operations Research Language Modelscited by 26
  6. 2025HeuriGym: An Agentic Benchmark for LLM-Crafted Heuristics in Combinatorial Optimizationcited by 19
  7. 2025Glia: A Human-Inspired AI for Automated Systems Design and Optimizationcited by 14
  8. 2025ARS: Automatic Routing Solver with Large Language Modelscited by 13
  9. 2025MTL-KD: Multi-Task Learning Via Knowledge Distillation for Generalizable Neural Vehicle Routing Solvercited by 13
  10. 2025HiFo-Prompt: Prompting with Hindsight and Foresight for LLM-based Automatic Heuristic Designcited by 13
  11. 2025Improving Generalization of Neural Combinatorial Optimization for Vehicle Routing Problems via Test-Time Projection Learningcited by 12
  12. 2025Learning to Insert for Constructive Neural Vehicle Routing Solvercited by 10
38 more
  1. 2025Optimizing Prompt Sequences using Monte Carlo Tree Search for LLM-Based Optimizationcited by 9
  2. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 9
  3. 2025Large Reasoning Models are not thinking straight: on the unreliability of thinking trajectoriescited by 8
  4. 2025Generalizable Heuristic Generation Through LLMs with Meta-Optimizationcited by 7
  5. 2025Rethinking Neural Combinatorial Optimization for Vehicle Routing Problems with Different Constraint Tightness Degreescited by 7
  6. 2025MOTIF: Multi-strategy Optimization via Turn-based Interactive Frameworkcited by 7
  7. 2025Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuningcited by 7
  8. 2025URS: A Unified Neural Routing Solver for Cross-Problem Zero-Shot Generalizationcited by 7
  9. 2026G-LNS: Generative Large Neighborhood Search for LLM-Based Automatic Heuristic Designcited by 7
  10. 2025Reasoning LLMs are Wandering Solution Explorerscited by 6
  11. 2025iDSE: Navigating Design Space Exploration in High-Level Synthesis Using LLMscited by 6
  12. 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 6
  13. 2025VRPAgent: LLM-Driven Discovery of Heuristic Operators for Vehicle Routing Problemscited by 6
  14. 2026EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Modelscited by 6
  15. 2025TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolutioncited by 5
  16. 2025An Agentic Framework with LLMs for Solving Complex Vehicle Routing Problemscited by 5
  17. 2026PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMscited by 5
  18. 2025LLM-Guided Search for Deletion-Correcting Codescited by 4
  19. 2025RedAHD: Reduction-Based End-to-End Automatic Heuristic Design with Large Language Modelscited by 4
  20. 2025Pareto-Grid-Guided Large Language Models for Fast and High-Quality Heuristics Design in Multi-Objective Combinatorial Optimizationcited by 4
  21. 2025Large Language Models in Operations Research: Methods, Applications, and Challengescited by 4
  22. 2025MetaMuse: Algorithm Generation via Creative Ideationcited by 4
  23. 2026LLM4Branch: Large Language Model for Discovering Efficient Branching Policies of Integer Programscited by 4
  24. 2025A Study on Leveraging Search and Self-Feedback for Agent Reasoningcited by 3
  25. 2025Refining Hybrid Genetic Search for CVRP via Reinforcement Learning-Finetuned LLMcited by 3
  26. 2026TIDE: Tuning-Integrated Dynamic Evolution for LLM-Based Automated Heuristic Designcited by 3
  27. 2026OptiML: An End-to-End Framework for Program Synthesis and CUDA Kernel Optimizationcited by 3
  28. 2026AHD Agent: Agentic Reinforcement Learning for Automatic Heuristic Designcited by 3
  29. 2026AlphaMemo: Structured Search-Process Memory for Self-Evolving Alpha Mining Agentscited by 3
  30. 2025ArGen: Auto-Regulation of Generative AI via GRPO and Policy-as-Codecited by 2
  31. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  32. 2026LLM Driven Design of Continuous Optimization Problems with Controllable High-level Propertiescited by 2
  33. 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priorscited by 2
  34. 2026APEX: Autonomous Policy Exploration for Self-Evolving LLM Agentscited by 2
  35. 2026LLM-Driven Co-Evolutionary Automated Heuristic Design for Bi-Component Coupled Combinatorial Optimizationcited by 2
  36. 2026RefineEvo: Planning-Guided Heuristic Evolution with Bidirectional Experiencecited by 2
  37. 2026Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirementscited by 2
  38. 2025A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Taskscited by 1

50 most cited of 100 All 100 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MASTER: A Multi-Agent System with LLM Specialized MCTS
  2. 2024A Training Data Recipe to Accelerate A* Search with Language Models
  3. 2024LiteSearch: Efficacious Tree Search for LLM
  4. 2025I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search
  5. 2025Complex LLM Planning via Automated Heuristics Discovery
  6. 2025LLM-First Search: Self-Guided Exploration of the Solution Space
  7. 2025Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code
  8. 2025PlanU: Large Language Model Reasoning through Planning under Uncertainty