# Relational Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1806.01830 · HTML: https://mlsanity.com/paper/1806.01830/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2018.

## Built on by (most cited first, 30 of 73)

- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Object-Region Video Transformers](https://mlsanity.com/paper/2110.06915/) (2021)
- [A Survey of Temporal Credit Assignment in Deep Reinforcement Learning](https://mlsanity.com/paper/2312.01072/) (2023)
- [Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction](https://mlsanity.com/paper/2409.18121/) (2024)
- [A Survey on Transformers in Reinforcement Learning](https://mlsanity.com/paper/2301.03044/) (2023)
- [Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction](https://mlsanity.com/paper/2306.01439/) (2023)
- [Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning](https://mlsanity.com/paper/2107.12544/) (2021)
- [Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero](https://mlsanity.com/paper/2310.16410/) (2023)
- [Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs](https://mlsanity.com/paper/2305.06343/) (2023)
- [Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking](https://mlsanity.com/paper/2102.09117/) (2021)
- [Procedural Generalization by Planning with Self-Supervised World Models](https://mlsanity.com/paper/2111.01587/) (2021)
- [Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applications](https://mlsanity.com/paper/2206.07922/) (2022)
- [Policy Architectures for Compositional Generalization in Control](https://mlsanity.com/paper/2203.05960/) (2022)
- [Systematic Visual Reasoning through Object-Centric Relational Abstraction](https://mlsanity.com/paper/2306.02500/) (2023)
- [Learning Efficient Multi-Agent Cooperative Visual Exploration](https://mlsanity.com/paper/2110.05734/) (2021)
- [Compositional Multi-Object Reinforcement Learning with Linear Relation Networks](https://mlsanity.com/paper/2201.13388/) (2022)
- [Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Exploration](https://mlsanity.com/paper/2301.03398/) (2023)
- [Learning to Optimize Industry-Scale Dynamic Pickup and Delivery Problems](https://mlsanity.com/paper/2105.12899/) (2021)
- [Hierarchical Imitation Learning with Vector Quantized Models](https://mlsanity.com/paper/2301.12962/) (2023)
- [Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approach](https://mlsanity.com/paper/2304.08349/) (2023)
- [On Efficient Reinforcement Learning for Full-length Game of StarCraft II](https://mlsanity.com/paper/2209.11553/) (2022)
- [PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Data](https://mlsanity.com/paper/2212.04821/) (2022)
- [Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models](https://mlsanity.com/paper/2407.15589/) (2024)
- [High-Accuracy Model-Based Reinforcement Learning, a Survey](https://mlsanity.com/paper/2107.08241/) (2021)
- [A Survey on Reinforcement Learning Methods in Character Animation](https://mlsanity.com/paper/2203.04735/) (2022)
- [Neuro-Symbolic Constraint Programming for Structured Prediction](https://mlsanity.com/paper/2103.17232/) (2021)
- [Explainability in reinforcement learning: perspective and position](https://mlsanity.com/paper/2203.11547/) (2022)
- [Learn the Time to Learn: Replay Scheduling in Continual Learning](https://mlsanity.com/paper/2209.08660/) (2022)

## Abstract (arXiv)

We introduce an approach for deep reinforcement learning (RL) that improves upon the efficiency, generalization capacity, and interpretability of conventional approaches through structured perception and relational reasoning. It uses self-attention to iteratively reason about the relations between entities in a scene and to guide a model-free policy. Our results show that in a novel navigation and planning task called Box-World, our agent finds interpretable solutions that improve upon baselines in terms of sample complexity, ability to generalize to more complex scenes than experienced during training, and overall performance. In the StarCraft II Learning Environment, our agent achieves state-of-the-art performance on six mini-games -- surpassing human grandmaster performance on four. By considering architectural inductive biases, our work opens new directions for overcoming important, but stubborn, challenges in deep RL.