# What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planning

arXiv: https://arxiv.org/abs/1901.01492 · HTML: https://mlsanity.com/paper/1901.01492/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2019.

## Built on by (most cited first, 10 of 10)

- [Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning](https://mlsanity.com/paper/2105.14074/) (2021)
- [Look Wide and Interpret Twice: Improving Performance on Interactive Instruction-following Tasks](https://mlsanity.com/paper/2106.00596/) (2021)
- [The Logical Options Framework](https://mlsanity.com/paper/2102.12571/) (2021)
- [Transformers are Adaptable Task Planners](https://mlsanity.com/paper/2207.02442/) (2022)
- [Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration](https://mlsanity.com/paper/2203.10033/) (2022)
- [Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation](https://mlsanity.com/paper/2212.11345/) (2022)
- [Multi-LLM QA with Embodied Exploration](https://mlsanity.com/paper/2406.10918/) (2024)
- [Learning to Build by Building Your Own Instructions](https://mlsanity.com/paper/2410.01111/) (2024)
- [SAGE: Generating Symbolic Goals for Myopic Models in Deep Reinforcement Learning](https://mlsanity.com/paper/2203.05079/) (2022)
- [Machine Intelligence that Understands Visual and Linguistic Information and Interacts with Humans and Environments](https://mlsanity.com/paper/2605.24020/) (2026)

## Abstract (arXiv)

Long-term planning poses a major difficulty to many reinforcement learning algorithms. This problem becomes even more pronounced in dynamic visual environments. In this work we propose Hierarchical Planning and Reinforcement Learning (HIP-RL), a method for merging the benefits and capabilities of Symbolic Planning with the learning abilities of Deep Reinforcement Learning. We apply HIPRL to the complex visual tasks of interactive question answering and visual semantic planning and achieve state-of-the-art results on three challenging datasets all while taking fewer steps at test time and training in fewer iterations. Sample results can be found at youtu.be/0TtWJ_0mPfI