# Reinforcement Learning with External Knowledge by using Logical Neural Networks

arXiv: https://arxiv.org/abs/2103.02363 · HTML: https://mlsanity.com/paper/2103.02363/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 10 of 10)

- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [TextWorld: A Learning Environment for Text-based Games](https://mlsanity.com/paper/1806.11532/) (2018)
- [Language Understanding for Text-based Games Using Deep Reinforcement Learning](https://mlsanity.com/paper/1506.08941/) (2015)
- [Logically-Constrained Reinforcement Learning](https://mlsanity.com/paper/1801.08099/) (2018)
- [Cautious Reinforcement Learning with Logical Constraints](https://mlsanity.com/paper/2002.12156/) (2020)
- [Counting to Explore and Generalize in Text-based Games](https://mlsanity.com/paper/1806.11525/) (2018)
- [Internal Model from Observations for Reward Shaping](https://mlsanity.com/paper/1806.01267/) (2018)
- [DAQN: Deep Auto-encoder and Q-Network](https://mlsanity.com/paper/1806.00630/) (2018)
- [Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning](https://mlsanity.com/paper/1810.01108/) (2018)

## Built on by (most cited first, 10 of 10)

- [Safe Reinforcement Learning via Probabilistic Logic Shields](https://mlsanity.com/paper/2303.03226/) (2023)
- [Neuro-Symbolic Reinforcement Learning with First-Order Logic](https://mlsanity.com/paper/2110.10963/) (2021)
- [Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language](https://mlsanity.com/paper/2205.06111/) (2022)
- [Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2307.01316/) (2023)
- [LOA: Logical Optimal Actions for Text-based Interaction Games](https://mlsanity.com/paper/2110.10973/) (2021)
- [Safe Reinforcement Learning in Black-Box Environments via Adaptive Shielding](https://mlsanity.com/paper/2405.18180/) (2024)
- [Commonsense Knowledge from Scene Graphs for Textual Environments](https://mlsanity.com/paper/2210.14162/) (2022)
- [Integrating Neurosymbolic AI in Advanced Air Mobility: A Comprehensive Survey](https://mlsanity.com/paper/2508.07163/) (2025)
- [DiffG-RL: Leveraging Difference between State and Common Sense](https://mlsanity.com/paper/2211.16002/) (2022)
- [The Ultimate Test of Superintelligent AI Agents: Can an AI Balance Care and Control in Asymmetric Relationships?](https://mlsanity.com/paper/2506.01813/) (2025)

## Abstract (arXiv)

Conventional deep reinforcement learning methods are sample-inefficient and usually require a large number of training trials before convergence. Since such methods operate on an unconstrained action set, they can lead to useless actions. A recent neuro-symbolic framework called the Logical Neural Networks (LNNs) can simultaneously provide key-properties of both neural networks and symbolic logic. The LNNs functions as an end-to-end differentiable network that minimizes a novel contradiction loss to learn interpretable rules. In this paper, we utilize LNNs to define an inference graph using basic logical operations, such as AND and NOT, for faster convergence in reinforcement learning. Specifically, we propose an integrated method that enables model-free reinforcement learning from external knowledge sources in an LNNs-based logical constrained framework such as action shielding and guide. Our results empirically demonstrate that our method converges faster compared to a model-free reinforcement learning method that doesn't have such logical constraints.