# Learning Generalized Reactive Policies using Deep Neural Networks

arXiv: https://arxiv.org/abs/1708.07280 · HTML: https://mlsanity.com/paper/1708.07280/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2017.

## Built on by (most cited first, 30 of 39)

- [BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimization](https://mlsanity.com/paper/2301.03313/) (2023)
- [Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limits](https://mlsanity.com/paper/2109.10129/) (2021)
- [Learning First-Order Representations for Planning from Black-Box States: New Results](https://mlsanity.com/paper/2105.10830/) (2021)
- [An Overview and Experimental Study of Learning-based Optimization Algorithms for Vehicle Routing Problem](https://mlsanity.com/paper/2107.07076/) (2021)
- [Learning Generalized Policies Without Supervision Using GNNs](https://mlsanity.com/paper/2205.06002/) (2022)
- [Return to Tradition: Learning Reliable Heuristics with Classical Machine Learning](https://mlsanity.com/paper/2403.16508/) (2024)
- [Guided Imitation of Task and Motion Planning](https://mlsanity.com/paper/2112.03386/) (2021)
- [Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width: Extended Version](https://mlsanity.com/paper/2203.14852/) (2022)
- [From Learning to Relearning: A Framework for Diminishing Bias in Social Robot Navigation](https://mlsanity.com/paper/2101.02647/) (2021)
- [Optimize Planning Heuristics to Rank, not to Estimate Cost-to-Goal](https://mlsanity.com/paper/2310.19463/) (2023)
- [A Novel Automated Curriculum Strategy to Solve Hard Sokoban Planning Instances](https://mlsanity.com/paper/2110.00898/) (2021)
- [Program Synthesis Guided Reinforcement Learning for Partially Observed Environments](https://mlsanity.com/paper/2102.11137/) (2021)
- [PG3: Policy-Guided Planning for Generalized Policy Generation](https://mlsanity.com/paper/2204.10420/) (2022)
- [Learning General Policies from Small Examples Without Supervision](https://mlsanity.com/paper/2101.00692/) (2021)
- [Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems](https://mlsanity.com/paper/2204.12665/) (2022)
- [Heuristic Search Planning with Deep Neural Networks using Imitation, Attention and Curriculum Learning](https://mlsanity.com/paper/2112.01918/) (2021)
- [Generalized Planning as Heuristic Search: A new planning search-space that leverages pointers over objects](https://mlsanity.com/paper/2301.11087/) (2023)
- [General Policies, Subgoal Structure, and Planning Width](https://mlsanity.com/paper/2311.05490/) (2023)
- [Exploiting Learned Policies in Focal Search](https://mlsanity.com/paper/2104.10535/) (2021)
- [Per-Domain Generalizing Policies: On Validation Instances and Scaling Behavior](https://mlsanity.com/paper/2505.00439/) (2025)
- [Learning General Policies with Policy Gradient Methods](https://mlsanity.com/paper/2512.19366/) (2025)
- [A* Search Without Expansions: Learning Heuristic Functions with Deep Q-Networks](https://mlsanity.com/paper/2102.04518/) (2021)
- [Meta-Reinforcement Learning for Heuristic Planning](https://mlsanity.com/paper/2107.02603/) (2021)
- [Hierarchical Reinforcement Learning with AI Planning Models](https://mlsanity.com/paper/2203.00669/) (2022)
- [Sample-efficient Iterative Lower Bound Optimization of Deep Reactive Policies for Planning in Continuous MDPs](https://mlsanity.com/paper/2203.12679/) (2022)
- [Learning Generalized Policy Automata for Relational Stochastic Shortest Path Problems](https://mlsanity.com/paper/2204.04301/) (2022)
- [A Differentiable Loss Function for Learning Heuristics in A*](https://mlsanity.com/paper/2209.05206/) (2022)
- [Exploration Policies for On-the-Fly Controller Synthesis: A Reinforcement Learning Approach](https://mlsanity.com/paper/2210.05393/) (2022)
- [Learning Search-Space Specific Heuristics Using Neural Networks](https://mlsanity.com/paper/2306.04019/) (2023)
- [Weisfeiler-Leman Features for Planning: A 1,000,000 Sample Size Hyperparameter Study](https://mlsanity.com/paper/2508.18515/) (2025)

## Abstract (arXiv)

We present a new approach to learning for planning, where knowledge acquired while solving a given set of planning problems is used to plan faster in related, but new problem instances. We show that a deep neural network can be used to learn and represent a generalized reactive policy (GRP) that maps a problem instance and a state to an action, and that the learned GRPs efficiently solve large classes of challenging problem instances. In contrast to prior efforts in this direction, our approach significantly reduces the dependence of learning on handcrafted domain knowledge or feature selection. Instead, the GRP is trained from scratch using a set of successful execution traces. We show that our approach can also be used to automatically learn a heuristic function that can be used in directed search algorithms. We evaluate our approach using an extensive suite of experiments on two challenging planning problem domains and show that our approach facilitates learning complex decision making policies and powerful heuristic functions with minimal human input. Videos of our results are available at goo.gl/Hpy4e3.