# SymDQN: Symbolic Knowledge and Reasoning in Neural Network-based Reinforcement Learning

arXiv: https://arxiv.org/abs/2504.02654 · HTML: https://mlsanity.com/paper/2504.02654/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Builds on (most cited first, 9 of 9)

- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [Analyzing Differentiable Fuzzy Logic Operators](https://mlsanity.com/paper/2002.06100/) (2020)
- [Logic Tensor Networks for Semantic Image Interpretation](https://mlsanity.com/paper/1705.08968/) (2017)
- [Induction and Exploitation of Subgoal Automata for Reinforcement Learning](https://mlsanity.com/paper/2009.03855/) (2020)
- [Faster-LTN: a neuro-symbolic, end-to-end object detection architecture](https://mlsanity.com/paper/2107.01877/) (2021)
- [Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach](https://mlsanity.com/paper/2402.19265/) (2024)
- [Injecting Prior Knowledge for Transfer Learning into Reinforcement Learning Algorithms using Logic Tensor Networks](https://mlsanity.com/paper/1906.06576/) (2019)
- [LTNtorch: PyTorch Implementation of Logic Tensor Networks](https://mlsanity.com/paper/2409.16045/) (2024)

## Abstract (arXiv)

We propose a learning architecture that allows symbolic control and guidance in reinforcement learning with deep neural networks. We introduce SymDQN, a novel modular approach that augments the existing Dueling Deep Q-Networks (DuelDQN) architecture with modules based on the neuro-symbolic framework of Logic Tensor Networks (LTNs). The modules guide action policy learning and allow reinforcement learning agents to display behaviour consistent with reasoning about the environment. Our experiment is an ablation study performed on the modules. It is conducted in a reinforcement learning environment of a 5x5 grid navigated by an agent that encounters various shapes, each associated with a given reward. The underlying DuelDQN attempts to learn the optimal behaviour of the agent in this environment, while the modules facilitate shape recognition and reward prediction. We show that our architecture significantly improves learning, both in terms of performance and the precision of the agent. The modularity of SymDQN allows reflecting on the intricacies and complexities of combining neural and symbolic approaches in reinforcement learning.