mlsanityData to 7 Oct 2026

BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Design

arXiv 2604.12898 ↗First posted 14 Apr 2026cs.AI math.COcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Model-based Hyper Heuristic (LHH) has recently emerged as an efficient way for automatic heuristic design. However, most existing LHHs just perform well in optimizing a single function within a pre-defined solver. Their single-layer evolution makes them not effective enough to write a competent complete solver. While some variants incorporate hyperparameter tuning or attempt to generate complex code through iterative local modifications, they still lack a high-level algorithmic modeling, leading to limited exploration efficiency. To address this, we reformulate heuristic design as a Bi-level Optimization problem and propose BEAM (Bi-level Memory-adaptive Algorithmic Evolution). BEAM's exterior layer evolves high-level algorithmic structures with function placeholders through genetic algorithm (GA), while the interior layer realizes these placeholders via Monte Carlo Tree Search (MCTS). We further introduce an Adaptive Memory module to facilitate complex code generation. To support the evaluation for complex code generation, we point out the limitations of starting LHHs from scratch or from code templates and introduce a Knowledge Augmentation (KA) Pipeline. Experimental results on several optimization problems demonstrate that BEAM significantly outperforms existing LHHs, notably reducing the optimality gap by 37.84% on aggregate in CVRP hybrid algorithm design. BEAM also designs a heuristic that outperforms SOTA Maximum Independent Set (MIS) solver KaMIS.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 72 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

39 of 72 references.

References · 39

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  5. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  7. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  8. 2024A Survey on Large Language Models for Code Generationcited by 646
  9. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  10. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  11. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 258
  12. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
27 more
  1. 2024OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Modelscited by 105
  2. 2019Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networkscited by 103
  3. 2024LLMOPT: Learning to Define and Solve General Optimization Problems from Scratchcited by 102
  4. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 100
  5. 2024LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristicscited by 92
  6. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 89
  7. 2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 74
  8. 2024HSEvo: Elevating Automatic Heuristic Design with Diversity-Driven Harmony Search and Genetic Algorithm Using LLMscited by 68
  9. 2024A Diffusion Model Framework for Unsupervised Neural Combinatorial Optimizationcited by 58
  10. 2020Cost-aware Bayesian Optimizationcited by 41
  11. 2023Zero Bubble Pipeline Parallelismcited by 39
  12. 2024GOAL: A Generalist Combinatorial Optimization Agent Learnercited by 39
  13. 2025Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learningcited by 39
  14. 2024LLM4AD: A Platform for Algorithm Design with Large Language Modelcited by 33
  15. 2015Finding Near-Optimal Independent Sets at Scalecited by 32
  16. 2024In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristicscited by 28
  17. 2023Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generationcited by 27
  18. 2025Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applicationscited by 26
  19. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  20. 2023Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm Representationcited by 21
  21. 2024Evolve Cost-aware Acquisition Functions Using Large Language Modelscited by 17
  22. 2025Planning of Heuristics: Strategic Planning on Large Language Models with Monte Carlo Tree Search for Automating Heuristic Optimizationcited by 13
  23. 2025Regularized Langevin Dynamics for Combinatorial Optimizationcited by 10
  24. 2024Bridging Large Language Models and Optimization: A Unified Framework for Text-attributed Combinatorial Optimizationcited by 8
  25. 2025Using Code Generation to Solve Open Instances of Combinatorial Design Problemscited by 4
  26. 2025MA-GTS: A Multi-Agent Framework for Solving Complex Graph Problems in Real-World Applicationscited by 3
  27. 2013KaHIP v3.00 -- Karlsruhe High Quality Partitioning -- User Guidecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026PACE: Primitive-Aware Code Evolution for Automated Algorithm Designcited by 0