# Neural Logic Reinforcement Learning

arXiv: https://arxiv.org/abs/1904.10729 · HTML: https://mlsanity.com/paper/1904.10729/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2019.

## Built on by (most cited first, 30 of 53)

- [A Survey on Interpretable Reinforcement Learning](https://mlsanity.com/paper/2112.13112/) (2021)
- [Neuro-Symbolic Artificial Intelligence: Current Trends](https://mlsanity.com/paper/2105.05330/) (2021)
- [Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction](https://mlsanity.com/paper/2306.01439/) (2023)
- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environments](https://mlsanity.com/paper/2306.08649/) (2023)
- [Neuro-Symbolic Hierarchical Rule Induction](https://mlsanity.com/paper/2112.13418/) (2021)
- [Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents](https://mlsanity.com/paper/2401.05821/) (2024)
- [LLMs for Relational Reasoning: How Far are We?](https://mlsanity.com/paper/2401.09042/) (2024)
- [Differentiable Logic Machines](https://mlsanity.com/paper/2102.11529/) (2021)
- [Neuro-Symbolic Forward Reasoning](https://mlsanity.com/paper/2110.09383/) (2021)
- [GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesis](https://mlsanity.com/paper/2205.13728/) (2022)
- [Multi-Step Deductive Reasoning Over Natural Language: An Empirical Study on Out-of-Distribution Generalisation](https://mlsanity.com/paper/2207.14000/) (2022)
- [Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approach](https://mlsanity.com/paper/2304.08349/) (2023)
- [BlendRL: A Framework for Merging Symbolic and Neural Policy Learning](https://mlsanity.com/paper/2410.11689/) (2024)
- [Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search](https://mlsanity.com/paper/2405.16450/) (2024)
- [PUZZLES: A Benchmark for Neural Algorithmic Reasoning](https://mlsanity.com/paper/2407.00401/) (2024)
- [MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees](https://mlsanity.com/paper/2209.07225/) (2022)
- [Explainable Reinforcement Learning via Model Transforms](https://mlsanity.com/paper/2209.12006/) (2022)
- [Learning Symbolic Rules for Interpretable Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08228/) (2021)
- [Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming](https://mlsanity.com/paper/2106.11417/) (2021)
- [What Planning Problems Can A Relational Neural Network Solve?](https://mlsanity.com/paper/2312.03682/) (2023)
- [Multimodal LLM-assisted Evolutionary Search for Programmatic Control Policies](https://mlsanity.com/paper/2508.05433/) (2025)
- [LOA: Logical Optimal Actions for Text-based Interaction Games](https://mlsanity.com/paper/2110.10973/) (2021)
- [Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Survey](https://mlsanity.com/paper/2304.12090/) (2023)
- [Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidence](https://mlsanity.com/paper/2402.02701/) (2024)
- [EXPIL: Explanatory Predicate Invention for Learning in Games](https://mlsanity.com/paper/2406.06107/) (2024)
- [Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimization](https://mlsanity.com/paper/2408.08761/) (2024)
- [Neural Meta-Symbolic Reasoning and Learning](https://mlsanity.com/paper/2211.11650/) (2022)
- [Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning](https://mlsanity.com/paper/2307.02689/) (2023)
- [Interpretable end-to-end Neurosymbolic Reinforcement Learning agents](https://mlsanity.com/paper/2410.14371/) (2024)

## Abstract (arXiv)

Deep reinforcement learning (DRL) has achieved significant breakthroughs in various tasks. However, most DRL algorithms suffer a problem of generalizing the learned policy which makes the learning performance largely affected even by minor modifications of the training environment. Except that, the use of deep neural networks makes the learned policies hard to be interpretable. To address these two challenges, we propose a novel algorithm named Neural Logic Reinforcement Learning (NLRL) to represent the policies in reinforcement learning by first-order logic. NLRL is based on policy gradient methods and differentiable inductive logic programming that have demonstrated significant advantages in terms of interpretability and generalisability in supervised tasks. Extensive experiments conducted on cliff-walking and blocks manipulation tasks demonstrate that NLRL can induce interpretable policies achieving near-optimal performance while demonstrating good generalisability to environments of different initial states and problem sizes.