# DRILL-- Deep Reinforcement Learning for Refinement Operators in ALC

arXiv: https://arxiv.org/abs/2106.15373 · HTML: https://mlsanity.com/paper/2106.15373/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 29 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [Towards Explainable Artificial Intelligence](https://mlsanity.com/paper/1909.12072/) (2019)
- [Multi-Hop Knowledge Graph Reasoning with Reward Shaping](https://mlsanity.com/paper/1808.10568/) (2018)
- [Reinforcement Knowledge Graph Reasoning for Explainable Recommendation](https://mlsanity.com/paper/1906.05237/) (2019)
- [Question Answering with Subgraph Embeddings](https://mlsanity.com/paper/1406.3676/) (2014)
- [Convolutional Complex Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.03130/) (2020)
- [Estimating Q(s,s') with Deep Deterministic Dynamics Gradients](https://mlsanity.com/paper/2002.09505/) (2020)
- [A shallow neural model for relation prediction](https://mlsanity.com/paper/2101.09090/) (2021)
- [A Holistic Natural Language Generation Framework for the Semantic Web](https://mlsanity.com/paper/1911.01248/) (2019)

## Built on by (most cited first, 3 of 3)

- [EvoLearner: Learning Description Logics with Evolutionary Algorithms](https://mlsanity.com/paper/2111.04879/) (2021)
- [Learning Permutation-Invariant Embeddings for Description Logic Concepts](https://mlsanity.com/paper/2303.01844/) (2023)
- [Explaining ELH Concept Descriptions through Counterfactual Reasoning](https://mlsanity.com/paper/2301.05109/) (2023)

## Abstract (arXiv)

Approaches based on refinement operators have been successfully applied to class expression learning on RDF knowledge graphs. These approaches often need to explore a large number of concepts to find adequate hypotheses. This need arguably stems from current approaches relying on myopic heuristic functions to guide their search through an infinite concept space. In turn, deep reinforcement learning provides effective means to address myopia by estimating how much discounted cumulated future reward states promise. In this work, we leverage deep reinforcement learning to accelerate the learning of concepts in ALC by proposing DRILL -- a novel class expression learning approach that uses a convolutional deep Q-learning model to steer its search. By virtue of its architecture, DRILL is able to compute the expected discounted cumulated future reward of more than 103 class expressions in a second on standard hardware. We evaluate DRILL on four benchmark datasets against state-of-the-art approaches. Our results suggest that DRILL converges to goal states at least 2.7× faster than state-of-the-art models on all benchmark datasets. We provide an open-source implementation of our approach, including training and evaluation scripts as well as pre-trained models.