# Leveraging Large Language Models to Develop Heuristics for Emerging Optimization Problems

arXiv: https://arxiv.org/abs/2503.03350 · HTML: https://mlsanity.com/paper/2503.03350/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 11 of 11)

- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model](https://mlsanity.com/paper/2401.02051/) (2024)
- [HSEvo: Elevating Automatic Heuristic Design with Diversity-Driven Harmony Search and Genetic Algorithm Using LLMs](https://mlsanity.com/paper/2412.14995/) (2024)
- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain](https://mlsanity.com/paper/2310.14053/) (2023)
- [Deep Learning Assisted Heuristic Tree Search for the Container Pre-marshalling Problem](https://mlsanity.com/paper/1709.09972/) (2017)
- [How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems](https://mlsanity.com/paper/2403.01757/) (2024)
- [Solving the unit-load pre-marshalling problem in block stacking storage systems with multiple access directions](https://mlsanity.com/paper/2207.09118/) (2022)
- [Sorting multibay block stacking storage systems](https://mlsanity.com/paper/2405.04847/) (2024)

## Built on by (most cited first, 6 of 6)

- [Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search](https://mlsanity.com/paper/2504.19636/) (2025)
- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [ACEvo: Adversarial Co-Evolution of Problem Distributions and Solvers for Combinatorial Optimization](https://mlsanity.com/paper/2506.02594/) (2025)
- [Using Reasoning Models to Generate Search Heuristics that Solve Open Instances of Combinatorial Design Problems](https://mlsanity.com/paper/2505.23881/) (2025)
- [Constraint-aware Path Planning from Natural Language Instructions Using Large Language Models](https://mlsanity.com/paper/2603.19257/) (2026)
- [LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization](https://mlsanity.com/paper/2609.34427/) (2026)

## Abstract (arXiv)

Combinatorial optimization problems often rely on heuristic algorithms to generate efficient solutions. However, the manual design of heuristics is resource-intensive and constrained by the designer's expertise. Recent advances in artificial intelligence, particularly large language models (LLMs), have demonstrated the potential to automate heuristic generation through evolutionary frameworks. Recent works focus only on well-known combinatorial optimization problems like the traveling salesman problem and online bin packing problem when designing constructive heuristics. This study investigates whether LLMs can effectively generate heuristics for niche, not yet broadly researched optimization problems, using the unit-load pre-marshalling problem as an example case. We propose the Contextual Evolution of Heuristics (CEoH) framework, an extension of the Evolution of Heuristics (EoH) framework, which incorporates problem-specific descriptions to enhance in-context learning during heuristic generation. Through computational experiments, we evaluate CEoH and EoH and compare the results. Results indicate that CEoH enables smaller LLMs to generate high-quality heuristics more consistently and even outperform larger models. Larger models demonstrate robust performance with or without contextualized prompts. The generated heuristics exhibit scalability to diverse instance configurations.