# Multi-objective Evolution of Heuristic Using Large Language Model

arXiv: https://arxiv.org/abs/2409.16867 · HTML: https://mlsanity.com/paper/2409.16867/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 41 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 7 of 7)

- [Attention, Learn to Solve Routing Problems!](https://mlsanity.com/paper/1803.08475/) (2018)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization](https://mlsanity.com/paper/2306.01102/) (2023)
- [LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery](https://mlsanity.com/paper/2405.09783/) (2024)
- [Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft](https://mlsanity.com/paper/2312.09238/) (2023)
- [tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)](https://mlsanity.com/paper/2402.02456/) (2024)

## Built on by (most cited first, 30 of 54)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design](https://mlsanity.com/paper/2501.08603/) (2025)
- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [LLM4AD: A Platform for Algorithm Design with Large Language Model](https://mlsanity.com/paper/2412.17287/) (2024)
- [HiFo-Prompt: Prompting with Hindsight and Foresight for LLM-based Automatic Heuristic Design](https://mlsanity.com/paper/2508.13333/) (2025)
- [Glia: A Human-Inspired AI for Automated Systems Design and Optimization](https://mlsanity.com/paper/2510.27176/) (2025)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search](https://mlsanity.com/paper/2504.19636/) (2025)
- [Generalizable Heuristic Generation Through LLMs with Meta-Optimization](https://mlsanity.com/paper/2505.20881/) (2025)
- [iDSE: Navigating Design Space Exploration in High-Level Synthesis Using LLMs](https://mlsanity.com/paper/2505.22086/) (2025)
- [Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings](https://mlsanity.com/paper/2506.07109/) (2025)
- [Improving Coherence and Persistence in Agentic AI for System Optimization](https://mlsanity.com/paper/2603.21321/) (2026)
- [Evolution of Optimization Algorithms for Global Placement via Large Language Models](https://mlsanity.com/paper/2504.17801/) (2025)
- [Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systems](https://mlsanity.com/paper/2505.15741/) (2025)
- [Fine-tuning Large Language Model for Automated Algorithm Design](https://mlsanity.com/paper/2507.10614/) (2025)
- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs](https://mlsanity.com/paper/2601.20539/) (2026)
- [Generative Design of Functional Metal Complexes Utilizing the Internal Knowledge of Large Language Models](https://mlsanity.com/paper/2410.18136/) (2024)
- [RedAHD: Reduction-Based End-to-End Automatic Heuristic Design with Large Language Models](https://mlsanity.com/paper/2505.20242/) (2025)
- [ACEvo: Adversarial Co-Evolution of Problem Distributions and Solvers for Combinatorial Optimization](https://mlsanity.com/paper/2506.02594/) (2025)
- [MeLA: A Metacognitive LLM-Driven Architecture for Automatic Heuristic Design](https://mlsanity.com/paper/2507.20541/) (2025)
- [Experience-Guided Reflective Co-Evolution of Prompts and Heuristics for Automatic Algorithm Design](https://mlsanity.com/paper/2509.24509/) (2025)
- [Rethinking Code Similarity for Automated Algorithm Design with LLMs](https://mlsanity.com/paper/2603.02787/) (2026)
- [From Understanding to Excelling: Template-Free Algorithm Design through Structural-Functional Co-Evolution](https://mlsanity.com/paper/2503.10721/) (2025)
- [Large Language Model-Driven Surrogate-Assisted Evolutionary Algorithm for Expensive Optimization](https://mlsanity.com/paper/2507.02892/) (2025)
- [Game-Theoretic Co-Evolution for LLM-Based Heuristic Discovery](https://mlsanity.com/paper/2601.22896/) (2026)
- [From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors](https://mlsanity.com/paper/2603.02792/) (2026)
- [LLM-Driven Co-Evolutionary Automated Heuristic Design for Bi-Component Coupled Combinatorial Optimization](https://mlsanity.com/paper/2606.00718/) (2026)
- [EvoSpeak: Large Language Models for Interpretable Genetic Programming-Evolved Heuristics](https://mlsanity.com/paper/2510.02686/) (2025)
- [MCCE: A Framework for Multi-LLM Collaborative Search in Discrete Spaces with Similarity-Filtered Preference Learning](https://mlsanity.com/paper/2510.06270/) (2025)

## Abstract (arXiv)

Heuristics are commonly used to tackle various search and optimization problems. Design heuristics usually require tedious manual crafting with domain knowledge. Recent works have incorporated Large Language Models (LLMs) into automatic heuristic search, leveraging their powerful language and coding capacity. However, existing research focuses on the optimal performance on the target problem as the sole objective, neglecting other criteria such as efficiency and scalability, which are vital in practice. To tackle this challenge, we propose to model the heuristic search as a multi-objective optimization problem and consider introducing additional practical criteria beyond optimal performance. Due to the complexity of the search space, conventional multi-objective optimization methods struggle to effectively handle LLM-based multi-objective heuristic search. We propose the first LLM-based multi-objective heuristic search framework, Multi-objective Evolution of Heuristic (MEoH), which integrates LLMs in a zero-shot manner to generate a non-dominated set of heuristics to meet multiple design criteria. We design a new dominance-dissimilarity mechanism for effective population management and selection, which incorporates both code dissimilarity in the search space and dominance in the objective space. MEoH is demonstrated in two well-known combinatorial optimization problems: the online Bin Packing Problem (BPP) and the Traveling Salesman Problem (TSP). The results indicate that a variety of elite heuristics are automatically generated in a single run, offering more trade-off options than the existing methods. It successfully achieves competitive or superior performance while improving efficiency up to 10 times. Moreover, we also observe that the multi-objective search introduces novel insights into heuristic design and leads to the discovery of diverse heuristics.