# EoH-S: Evolution of Heuristic Set using LLMs for Automated Heuristic Design

arXiv: https://arxiv.org/abs/2508.03082 · HTML: https://mlsanity.com/paper/2508.03082/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 50 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 13 of 13)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [Generalize a Small Pre-trained Model to Arbitrarily Large TSP Instances](https://mlsanity.com/paper/2012.10658/) (2020)
- [Neural Combinatorial Optimization with Heavy Decoder: Toward Large Scale Generalization](https://mlsanity.com/paper/2310.07985/) (2023)
- [Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design](https://mlsanity.com/paper/2501.08603/) (2025)
- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [HSEvo: Elevating Automatic Heuristic Design with Diversity-Driven Harmony Search and Genetic Algorithm Using LLMs](https://mlsanity.com/paper/2412.14995/) (2024)
- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [CALM: Co-evolution of Algorithms and Language Model for Automatic Heuristic Design](https://mlsanity.com/paper/2505.12285/) (2025)
- [On the Generalization of Neural Combinatorial Optimization Heuristics](https://mlsanity.com/paper/2206.00787/) (2022)
- [Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing](https://mlsanity.com/paper/2501.11411/) (2025)
- [Prompt Learning for Generalized Vehicle Routing](https://mlsanity.com/paper/2405.12262/) (2024)

## Built on by (most cited first, 30 of 36)

- [CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization](https://mlsanity.com/paper/2510.14150/) (2025)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [VRPAgent: LLM-Driven Discovery of Heuristic Operators for Vehicle Routing Problems](https://mlsanity.com/paper/2510.07073/) (2025)
- [G-LNS: Generative Large Neighborhood Search for LLM-Based Automatic Heuristic Design](https://mlsanity.com/paper/2602.08253/) (2026)
- [PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs](https://mlsanity.com/paper/2601.20539/) (2026)
- [Automated Discovery Has No Universally Superior Harness](https://mlsanity.com/paper/2607.18235/) (2026)
- [TIDE: Tuning-Integrated Dynamic Evolution for LLM-Based Automated Heuristic Design](https://mlsanity.com/paper/2601.21239/) (2026)
- [AHD Agent: Agentic Reinforcement Learning for Automatic Heuristic Design](https://mlsanity.com/paper/2605.08756/) (2026)
- [MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution](https://mlsanity.com/paper/2607.22832/) (2026)
- [Game-Theoretic Co-Evolution for LLM-Based Heuristic Discovery](https://mlsanity.com/paper/2601.22896/) (2026)
- [LLM-Driven Co-Evolutionary Automated Heuristic Design for Bi-Component Coupled Combinatorial Optimization](https://mlsanity.com/paper/2606.00718/) (2026)
- [Algorithmic Prompt-Augmentation for Efficient LLM-Based Heuristic Design for A* Search](https://mlsanity.com/paper/2601.19622/) (2026)
- [Breaking Validity-Induced Boundaries to Expand Algorithm Search Space: A Two-Stage AST-Based Operator for LLM-Driven Automated Heuristic Evolution](https://mlsanity.com/paper/2604.16420/) (2026)
- [SpecAHD: Localize to Specialize for Automated Heuristic Design in Large-Scale Routing Problems](https://mlsanity.com/paper/2607.23676/) (2026)
- [Beyond Average Performance: Dynamic Instance Clustering and Specialized Algorithm Design in LLM-Assisted Evolutionary Search](https://mlsanity.com/paper/2608.03129/) (2026)
- [MuEvo: LLM-Driven Evolution of Multi-Heuristic Ensemble](https://mlsanity.com/paper/2608.03636/) (2026)
- [ATLAS: Scaffold-Free Algorithm Synthesis by LLMs via Embedding-Guided Quality-Diversity Search](https://mlsanity.com/paper/2608.15546/) (2026)
- [CoupleEvo: Evolving Heuristics for Coupled Optimization Problems Using Large Language Models](https://mlsanity.com/paper/2605.06341/) (2026)
- [LLMSYS-HPOBench: Hyperparameter Optimization Benchmark Suite for Real-World LLM Systems](https://mlsanity.com/paper/2605.08305/) (2026)
- [MeEvo: Metacognitive Evolution Combined with Natural Evolution for Automatic Heuristic Design](https://mlsanity.com/paper/2606.14202/) (2026)
- [RAISE: LLM-based Automated Heuristic Design with Robust Adversary Instance Search](https://mlsanity.com/paper/2606.31801/) (2026)
- [RELIC: Revealed Principles for Learning Interpretable Composable Skills in Multi-Agent Planning](https://mlsanity.com/paper/2607.16745/) (2026)
- [LLM-Guided Evolutionary Search for Constraint Model Reformulation to Improve Solver Efficiency](https://mlsanity.com/paper/2607.28268/) (2026)
- [PACE: Primitive-Aware Code Evolution for Automated Algorithm Design](https://mlsanity.com/paper/2608.07395/) (2026)
- [MOSAIC: Adversarial Co-evolution of Specialist Heuristics and Problem Instances for LLM-based Automated Heuristic Design](https://mlsanity.com/paper/2608.07544/) (2026)
- [ES-AHD: An Evolution Strategy Framework for Automatic Heuristic Design](https://mlsanity.com/paper/2609.00023/) (2026)
- [LLM-Driven Algorithm Design for Quantum Circuit Synthesis based on Binary Decision Diagrams](https://mlsanity.com/paper/2609.05327/) (2026)
- [RouteRepair: Instance-Level Failure Diagnosis and Targeted Repair in LLM-Based Automated Heuristic Design for Routing Optimization](https://mlsanity.com/paper/2609.11452/) (2026)
- [T-GADE: Thermodynamical Generative-AI-Driven Evolution of LLM Artifacts](https://mlsanity.com/paper/2609.12286/) (2026)
- [PINNForge: Execution-Grounded Evolutionary Design of Physics-Informed Neural Networks](https://mlsanity.com/paper/2609.23023/) (2026)

## Abstract (arXiv)

Automated Heuristic Design (AHD) using Large Language Models (LLMs) has achieved notable success in recent years. Despite the effectiveness of existing approaches, they only design a single heuristic to serve all problem instances, often inducing poor generalization across different distributions or settings. To address this issue, we propose Automated Heuristic Set Design (AHSD), a new formulation for LLM-driven AHD. The aim of AHSD is to automatically generate a small-sized complementary heuristic set to serve diverse problem instances, such that each problem instance could be optimized by at least one heuristic in this set. We show that the objective function of AHSD is monotone and supermodular. Then, we propose Evolution of Heuristic Set (EoH-S) to apply the AHSD formulation for LLM-driven AHD. With two novel mechanisms of complementary population management and complementary-aware memetic search, EoH-S could effectively generate a set of high-quality and complementary heuristics. Comprehensive experimental results on three AHD tasks with diverse instances spanning various sizes and distributions demonstrate that EoH-S consistently outperforms existing state-of-the-art AHD methods and achieves up to 60% performance improvements.