# Reinforcement Learning Driven Heuristic Optimization

arXiv: https://arxiv.org/abs/1906.06639 · HTML: https://mlsanity.com/paper/1906.06639/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2019.

## Built on by (most cited first, 12 of 12)

- [Generalization in Deep RL for TSP Problems via Equivariance and Local Search](https://mlsanity.com/paper/2110.03595/) (2021)
- [One model Packs Thousands of Items with Recurrent Conditional Query Learning](https://mlsanity.com/paper/2111.06726/) (2021)
- [AI/ML Algorithms and Applications in VLSI Design and Technology](https://mlsanity.com/paper/2202.10015/) (2022)
- [Fast ML-driven Analog Circuit Layout using Reinforcement Learning and Steiner Trees](https://mlsanity.com/paper/2405.16951/) (2024)
- [Machine Learning for the Multi-Dimensional Bin Packing Problem: Literature Review and Empirical Evaluation](https://mlsanity.com/paper/2312.08103/) (2023)
- [Combinatorial Optimization Augmented Machine Learning](https://mlsanity.com/paper/2601.10583/) (2026)
- [Neural Simulated Annealing](https://mlsanity.com/paper/2203.02201/) (2022)
- [Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms](https://mlsanity.com/paper/2504.06126/) (2025)
- [Learning Virtual Machine Scheduling in Cloud Computing through Language Agents](https://mlsanity.com/paper/2505.10117/) (2025)
- [Towards VM Rescheduling Optimization Through Deep Reinforcement Learning](https://mlsanity.com/paper/2505.17359/) (2025)
- [DOPPLER: Dual-Policy Learning for Device Assignment in Asynchronous Dataflow Graphs](https://mlsanity.com/paper/2505.23131/) (2025)
- [Synergizing Reinforcement Learning and Genetic Algorithms for Neural Combinatorial Optimization](https://mlsanity.com/paper/2506.09404/) (2025)

## Abstract (arXiv)

Heuristic algorithms such as simulated annealing, Concorde, and METIS are effective and widely used approaches to find solutions to combinatorial optimization problems. However, they are limited by the high sample complexity required to reach a reasonable solution from a cold-start. In this paper, we introduce a novel framework to generate better initial solutions for heuristic algorithms using reinforcement learning (RL), named RLHO. We augment the ability of heuristic algorithms to greedily improve upon an existing initial solution generated by RL, and demonstrate novel results where RL is able to leverage the performance of heuristics as a learning signal to generate better initialization. We apply this framework to Proximal Policy Optimization (PPO) and Simulated Annealing (SA). We conduct a series of experiments on the well-known NP-complete bin packing problem, and show that the RLHO method outperforms our baselines. We show that on the bin packing problem, RL can learn to help heuristics perform even better, allowing us to combine the best parts of both approaches.