# Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming

arXiv: https://arxiv.org/abs/2106.11417 · HTML: https://mlsanity.com/paper/2106.11417/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 29 of 29)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [Model-Ensemble Trust-Region Policy Optimization](https://mlsanity.com/paper/1802.10592/) (2018)
- [Learning Explanatory Rules from Noisy Data](https://mlsanity.com/paper/1711.04574/) (2017)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Explainable Machine Learning for Scientific Insights and Discoveries](https://mlsanity.com/paper/1905.08883/) (2019)
- [End-to-End Differentiable Proving](https://mlsanity.com/paper/1705.11040/) (2017)
- [Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning](https://mlsanity.com/paper/1905.06088/) (2019)
- [Neural Logic Machines](https://mlsanity.com/paper/1904.11694/) (2019)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Hierarchical Imitation and Reinforcement Learning](https://mlsanity.com/paper/1803.00590/) (2018)
- [Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis](https://mlsanity.com/paper/1805.04276/) (2018)
- [SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning](https://mlsanity.com/paper/1811.00090/) (2018)
- [Learning programs by learning from failures](https://mlsanity.com/paper/2005.02259/) (2020)
- [Neural Logic Reinforcement Learning](https://mlsanity.com/paper/1904.10729/) (2019)
- [PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making](https://mlsanity.com/paper/1804.07779/) (2018)
- [Turning 30: New Ideas in Inductive Logic Programming](https://mlsanity.com/paper/2002.11002/) (2020)
- [Learning Dexterous Manipulation Policies from Experience and Imitation](https://mlsanity.com/paper/1611.05095/) (2016)
- ["Dave...I can assure you...that it's going to be all right..." -- A definition, case for, and survey of algorithmic assurances in human-autonomy trust relationships](https://mlsanity.com/paper/1711.03846/) (2017)
- [Learning Algorithms via Neural Logic Networks](https://mlsanity.com/paper/1904.01554/) (2019)
- [Hierarchical Reinforcement Learning with Abductive Planning](https://mlsanity.com/paper/1806.10792/) (2018)
- [Robot Representation and Reasoning with Knowledge from Reinforcement Learning](https://mlsanity.com/paper/1809.11074/) (2018)
- [On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulation](https://mlsanity.com/paper/1911.08927/) (2019)

## Built on by (most cited first, 8 of 8)

- [Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction](https://mlsanity.com/paper/2306.01439/) (2023)
- [Structure in Deep Reinforcement Learning: A Survey and Open Problems](https://mlsanity.com/paper/2306.16021/) (2023)
- [Explainability in Deep Reinforcement Learning, a Review into Current Methods and Applications](https://mlsanity.com/paper/2207.01911/) (2022)
- [Interpretability in Action: Exploratory Analysis of VPT, a Minecraft Agent](https://mlsanity.com/paper/2407.12161/) (2024)
- [A Survey of Reinforcement Learning for Optimization in Automation](https://mlsanity.com/paper/2502.09417/) (2025)
- [TAG: A Decentralized Framework for Multi-Agent Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2502.15425/) (2025)
- [Interpretable Hidden Markov Model-Based Deep Reinforcement Learning Hierarchical Framework for Predictive Maintenance of Turbofan Engines](https://mlsanity.com/paper/2206.13433/) (2022)
- [Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery](https://mlsanity.com/paper/2311.14270/) (2023)

## Abstract (arXiv)

Recently deep reinforcement learning has achieved tremendous success in wide ranges of applications. However, it notoriously lacks data-efficiency and interpretability. Data-efficiency is important as interacting with the environment is expensive. Further, interpretability can increase the transparency of the black-box-style deep RL models and hence gain trust from the users. In this work, we propose a new hierarchical framework via symbolic RL, leveraging a symbolic transition model to improve the data-efficiency and introduce the interpretability for learned policy. This framework consists of a high-level agent, a subtask solver and a symbolic transition model. Without assuming any prior knowledge on the state transition, we adopt inductive logic programming (ILP) to learn the rules of symbolic state transitions, introducing interpretability and making the learned behavior understandable to users. In empirical experiments, we confirmed that the proposed framework offers approximately between 30% to 40% more data efficiency over previous methods.