# Reinforcement Learning with A* and a Deep Heuristic

arXiv: https://arxiv.org/abs/1811.07745 · HTML: https://mlsanity.com/paper/1811.07745/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2018.

## Built on by (most cited first, 4 of 4)

- [Reinforcement Learning based Negotiation-aware Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2107.03600/) (2021)
- [Vanishing Bias Heuristic-guided Reinforcement Learning Algorithm](https://mlsanity.com/paper/2306.10216/) (2023)
- [Towards Learning Scalable Agile Dynamic Motion Planning for Robosoccer Teams with Policy Optimization](https://mlsanity.com/paper/2502.05526/) (2025)
- [AI planning in the imagination: High-level planning on learned abstract search spaces](https://mlsanity.com/paper/2308.08693/) (2023)

## Abstract (arXiv)

A* is a popular path-finding algorithm, but it can only be applied to those domains where a good heuristic function is known. Inspired by recent methods combining Deep Neural Networks (DNNs) and trees, this study demonstrates how to train a heuristic represented by a DNN and combine it with A*. This new algorithm which we call aleph-star can be used efficiently in domains where the input to the heuristic could be processed by a neural network. We compare aleph-star to N-Step Deep Q-Learning (DQN Mnih et al. 2013) in a driving simulation with pixel-based input, and demonstrate significantly better performance in this scenario.