# Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems

arXiv: https://arxiv.org/abs/2204.12665 · HTML: https://mlsanity.com/paper/2204.12665/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 45 references that are in the map.
- First posted on arXiv 27 Apr 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Learning Generalized Reactive Policies using Deep Neural Networks](https://mlsanity.com/paper/1708.07280/) (2017)
- [Learning Features and Abstract Actions for Computing Generalized Plans](https://mlsanity.com/paper/1811.07231/) (2018)
- [Learning Curriculum Policies for Reinforcement Learning](https://mlsanity.com/paper/1812.00285/) (2018)
- [Learning Generalized Relational Heuristic Networks for Model-Agnostic Planning](https://mlsanity.com/paper/2007.06702/) (2020)
- [Size Independent Neural Transfer for RDDL Planning](https://mlsanity.com/paper/1902.03081/) (2019)
- [Fitted Q-Learning for Relational Domains](https://mlsanity.com/paper/2006.05595/) (2020)

## Built on by (most cited first, 7 of 7)

- [Structure in Deep Reinforcement Learning: A Survey and Open Problems](https://mlsanity.com/paper/2306.16021/) (2023)
- [Hierarchical Decomposition and Analysis for Generalized Planning](https://mlsanity.com/paper/2212.02823/) (2022)
- [Analyzing Generalization in Policy Networks: A Case Study with the Double-Integrator System](https://mlsanity.com/paper/2312.10472/) (2023)
- [Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings](https://mlsanity.com/paper/2402.08145/) (2024)
- [The Price Is Not Right: Neuro-Symbolic Methods Outperform VLAs on Structured Long-Horizon Manipulation Tasks with Significantly Lower Energy Consumption](https://mlsanity.com/paper/2602.19260/) (2026)
- [Learning Dynamic Abstract Representations for Sample-Efficient Reinforcement Learning](https://mlsanity.com/paper/2210.01955/) (2022)
- [Learning to Search and Searching to Learn for Generalization in Planning](https://mlsanity.com/paper/2605.25720/) (2026)

## Abstract (arXiv)

Reinforcement learning in problems with symbolic state spaces is challenging due to the need for reasoning over long horizons. This paper presents a new approach that utilizes relational abstractions in conjunction with deep learning to learn a generalizable Q-function for such problems. The learned Q-function can be efficiently transferred to related problems that have different object names and object quantities, and thus, entirely different state spaces. We show that the learned generalized Q-function can be utilized for zero-shot transfer to related problems without an explicit, hand-coded curriculum. Empirical evaluations on a range of problems show that our method facilitates efficient zero-shot transfer of learned knowledge to much larger problem instances containing many objects.