# Robust Heuristic Algorithm Design with LLMs

arXiv: https://arxiv.org/abs/2510.08755 · HTML: https://mlsanity.com/paper/2510.08755/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 66 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Long-context LLMs Struggle with Long In-context Learning](https://mlsanity.com/paper/2404.02060/) (2024)
- [DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://mlsanity.com/paper/2006.08381/) (2020)
- [Themis: Fair and Efficient GPU Cluster Scheduling](https://mlsanity.com/paper/1907.01484/) (2019)
- [TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches](https://mlsanity.com/paper/2111.04867/) (2021)
- [AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement](https://mlsanity.com/paper/2412.06176/) (2024)
- [Synthesizing Optimal Collective Algorithms](https://mlsanity.com/paper/2008.08708/) (2020)
- [What do LLMs need to Synthesize Correct Router Configurations?](https://mlsanity.com/paper/2307.04945/) (2023)
- [CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters](https://mlsanity.com/paper/2308.00852/) (2023)
- [GAP: Generalizable Approximate Graph Partitioning Framework](https://mlsanity.com/paper/1903.00614/) (2019)
- [Solving Large-Scale Granular Resource Allocation Problems Efficiently with POP](https://mlsanity.com/paper/2110.11927/) (2021)
- [Systematic Evaluation of Long-Context LLMs on Financial Concepts](https://mlsanity.com/paper/2412.15386/) (2024)
- [Efficient Direct-Connect Topologies for Collective Communications](https://mlsanity.com/paper/2202.03356/) (2022)
- [Solving Max-Min Fair Resource Allocations Quickly on Large Graphs](https://mlsanity.com/paper/2310.09699/) (2023)
- [Finding Adversarial Inputs for Heuristics using Multi-level Optimization](https://mlsanity.com/paper/2311.12779/) (2023)
- [Everything Matters in Programmable Packet Scheduling](https://mlsanity.com/paper/2308.00797/) (2023)

## Built on by (most cited first, 3 of 3)

- [Vulcan: Instance-specialized, Verifiable Systems Heuristics Through LLM-driven Search](https://mlsanity.com/paper/2512.25065/) (2025)
- [RAISE: LLM-based Automated Heuristic Design with Robust Adversary Instance Search](https://mlsanity.com/paper/2606.31801/) (2026)
- [ReForge: Keeping ABR Algorithms Never Finished with Verified Large Language Model Edits](https://mlsanity.com/paper/2608.15138/) (2026)

## Abstract (arXiv)

We posit that we can generate more robust and performant heuristics if we augment approaches using LLMs for heuristic design with tools that explain why heuristics underperform and suggestions about how to fix them. We find even simple ideas that (1) expose the LLM to instances where the heuristic underperforms; (2) explain why they occur; and (3) specialize design to regions in the input space, can produce more robust algorithms compared to existing techniques~ -- ~the heuristics we produce have a ~28× better worst-case performance compared to FunSearch, improve average performance, and maintain the runtime.