# Improving Existing Optimization Algorithms with LLMs

arXiv: https://arxiv.org/abs/2502.08298 · HTML: https://mlsanity.com/paper/2502.08298/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 17 of 17)

- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [Large Language Models Meet NL2Code: A Survey](https://mlsanity.com/paper/2212.09420/) (2022)
- [Self-planning Code Generation with Large Language Models](https://mlsanity.com/paper/2303.06689/) (2023)
- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [Improving Code Generation by Training with Natural Language Feedback](https://mlsanity.com/paper/2303.16749/) (2023)
- [A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages](https://mlsanity.com/paper/2410.03981/) (2024)
- [Large Language Models for Compiler Optimization](https://mlsanity.com/paper/2309.07062/) (2023)
- [A Survey on Evaluating Large Language Models in Code Generation Tasks](https://mlsanity.com/paper/2408.16498/) (2024)
- [Large Language Model-Aware In-Context Learning for Code Generation](https://mlsanity.com/paper/2310.09748/) (2023)
- [Evolving Code with A Large Language Model](https://mlsanity.com/paper/2401.07102/) (2024)
- [LLM4AD: A Platform for Algorithm Design with Large Language Model](https://mlsanity.com/paper/2412.17287/) (2024)
- [Prototype2Code: End-to-end Front-end Code Generation from UI Design Prototypes](https://mlsanity.com/paper/2405.04975/) (2024)
- [Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing](https://mlsanity.com/paper/2501.11411/) (2025)
- [Grounding Data Science Code Generation with Input-Output Specifications](https://mlsanity.com/paper/2402.08073/) (2024)

## Built on by (most cited first, 4 of 4)

- [Combinatorial Optimization for All: Using LLMs to Aid Non-Experts in Improving Optimization Algorithms](https://mlsanity.com/paper/2503.10968/) (2025)
- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [ACEvo: Adversarial Co-Evolution of Problem Distributions and Solvers for Combinatorial Optimization](https://mlsanity.com/paper/2506.02594/) (2025)
- [LLM-Based Instance-Driven Heuristic Bias In the Context of a Biased Random Key Genetic Algorithm](https://mlsanity.com/paper/2509.09707/) (2025)

## Abstract (arXiv)

The integration of Large Language Models (LLMs) into optimization has created a powerful synergy, opening exciting research opportunities. This paper investigates how LLMs can enhance existing optimization algorithms. Using their pre-trained knowledge, we demonstrate their ability to propose innovative heuristic variations and implementation strategies. To evaluate this, we applied a non-trivial optimization algorithm, Construct, Merge, Solve and Adapt (CMSA) -- a hybrid metaheuristic for combinatorial optimization problems that incorporates a heuristic in the solution construction phase. Our results show that an alternative heuristic proposed by GPT-4o outperforms the expert-designed heuristic of CMSA, with the performance gap widening on larger and denser graphs. Project URL: https://imp-opt-algo-llms.surge.sh/